Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Add FEATURE knob for testing for UFS extended attribute kernel support

Support can be verified via `feature_present("ufs_extattr")`, etc.

Differential Revision: https://reviews.freebsd.org/D6053
MFC after: 2 weeks
Relnotes: yes
Reviewed by: asomers, kib
Sponsored by: EMC / Isilon Storage Division

Remove some NULL checks for M_WAITOK allocations.

MFC after: 1 month
Sponsored by: The FreeBSD Foundation

Don't assume that UFS on-disk format of a directory is the same as
defined by <sys/dirent.h>

Always start parsing at DIRBLKSIZ aligned offset, skip first entries if
uio_offset is not DIRBLKSIZ aligned. Return EINVAL if buffer is too
small for single entry.

Preallocate buffer for cookies. Cookies will be replaced with d_off
field in struct dirent at later point.

Skip entries with zero inode number.

Stop mangling dirent in ufs_extattr_iterate_directory().

Reviewed by: kib
Sponsored by: Google Summer Of Code 2011

Change i_gen in UFS to an unsigned type.

Missed format specifier.

Reported by: mdf
MFC after: 4 weeks

Change i_gen in UFS to an unsigned type.

In UFS, i_gen is a random generated value and there is not way for
it to be negative. Actually, the value of i_gen is just used to
match bit patterns and it is of not consequence if the values are
signed or not.

Following other filesystems, set it to unsigned and use it as such,

Discussed by: mckusick
Reviewed by: mckusick (previous version)
MFC after: 4 weeks

The r241025 fixed the case when a binary, executed from nullfs mount,
was still possible to open for write from the lower filesystem. There
is a symmetric situation where the binary could already has file
descriptors opened for write, but it can be executed from the nullfs
overlay.

Handle the issue by passing one v_writecount reference to the lower
vnode if nullfs vnode has non-zero v_writecount. Note that only one
write reference can be donated, since nullfs only keeps one use
reference on the lower vnode. Always use the lower vnode v_writecount
for the checks.

Introduce the VOP_GET_WRITECOUNT to read v_writecount, which is
currently always bypassed to the lower vnode, and VOP_ADD_WRITECOUNT
to manipulate the v_writecount value, which manages a single bypass
reference to the lower vnode. Caling the VOPs instead of directly
accessing v_writecount provide the fix described in the previous
paragraph.

Tested by: pho
MFC after: 3 weeks

Remove unused thread argument from ufs_extattr_uepm_lock()/ufs_extattr_uepm_unlock().

Add KTR_VFS traces to track modifications to a vnode's writecount.

Fix lock leak.

Reported by: Alex Lyashkov
Approved by: re (kib)
MFC after: 1 week

Remove the thread argument from the FSD (File-System Dependent) parts of
the VFS. Now all the VFS_* functions and relating parts don't want the
context as long as it always refers to curthread.

In some points, in particular when dealing with VOPs and functions living
in the same namespace (eg. vflush) which still need to be converted,
pass curthread explicitly in order to retain the old behaviour.
Such loose ends will be fixed ASAP.

While here fix a bug: now, UFS_EXTATTR can be compiled alone without the
UFS_EXTATTR_AUTOSTART option.

VFS KPI is heavilly changed by this commit so thirdy parts modules needs
to be recompiled. Bump __FreeBSD_version in order to signal such
situation.

Lock the uepm_lock around the autostart of extattrs.

Reported and tested by: pho
Reviewed by: rwatson
MFC after: 3 weeks

Fix a number of style issues in the MALLOC / FREE commit. I've tried to
be careful not to fix anything that was already broken; the NFSv4 code is
particularly bad in this respect.

Retire the MALLOC and FREE macros. They are an abomination unto style(9).

MFC after: 3 months

When calling extattr_check_cred, use V{READ,WRITE}, not I{READ,WRITE}.

Approved by: rwatson (mentor)

Continue on-going campaign to replace lockmgr locks with sx locks where
the specific semantics of ockmgr aren't required: update UFS1 extended
attributes to protect its data structures using an sx lock.

While here, update comments on lock granularity.

MFC after: 2 weeks

Don't auto-start or allow extattrctl for UFS2 file systems, as UFS2 has
native extended attributes. This didn't interfere with the operation of
UFS2 extended attributes, but the code shouldn't be running for UFS2.

MFC after: 2 weeks

Cleanup lockmgr interface and exported KPI:
- Remove the "thread" argument from the lockmgr() function as it is
always curthread now
- Axe lockcount() function as it is no longer used
- Axe LOCKMGR_ASSERT() as it is bogus really and no currently used.
Hopefully this will be soonly replaced by something suitable for it.
- Remove the prototype for dumplockinfo() as the function is no longer
present

Addictionally:
- Introduce a KASSERT() in lockstatus() in order to let it accept only
curthread or NULL as they should only be passed
- Do a little bit of style(9) cleanup on lockmgr.h

KPI results heavilly broken by this change, so manpages and
FreeBSD_version will be modified accordingly by further commits.

Tested by: matteo

VOP_LOCK1() (and so VOP_LOCK()) and VOP_UNLOCK() are only used in
conjuction with 'thread' argument passing which is always curthread.
Remove the unuseful extra-argument and pass explicitly curthread to lower
layer functions, when necessary.

KPI results broken by this change, which should affect several ports, so
version bumping and manpage update will be further committed.

Tested by: kris, pho, Diego Sardina <siarodx at gmail dot com>

vn_lock() is currently only used with the 'curthread' passed as argument.
Remove this argument and pass curthread directly to underlying
VOP_LOCK1() VFS method. This modify makes the code cleaner and in
particular remove an annoying dependence helping next lockmgr() cleanup.
KPI results, obviously, changed.

Manpage and FreeBSD_version will be updated through further commits.

As a side note, would be valuable to say that next commits will address
a similar cleanup about VFS methods, in particular vop_lock1 and
vop_unlock.

Tested by: Diego Sardina <siarodx at gmail dot com>,
Andrea Di Pasquale <whyx dot it at gmail dot com>

Change the VOP_OPEN(), vn_open() vnode operation and d_fdopen() cdev operation
argument from being file descriptor index into the pointer to struct file:
part 2. Convert calls missed in the first big commit.

Noted by: rwatson
Pointy hat to: kib

Sweep kernel replacing suser(9) calls with priv(9) calls, assigning
specific privilege names to a broad range of privileges. These may
require some future tweaking.

Sponsored by: nCircle Network Security, Inc.
Obtained from: TrustedBSD Project
Discussed on: arch@
Reviewed (at least in part) by: mlaier, jmg, pjd, bde, ceri,
Alex Lyashkov <umka at sevcity dot net>,
Skip Ford <skip dot ford at verizon dot net>,
Antoine Brodin <antoine dot brodin at laposte dot net>

- Reorder calls to vrele() after calls to vput() when the vrele is a
directory. vrele() may lock the passed vnode, which in these cases would
give an invalid lock order of child -> parent. These situations are
deadlock prone although do not typically deadlock because the vrele
is typically not releasing the last reference to the vnode. Users of
vrele must consider it as a call to vn_lock() and order it appropriately.

MFC After: 1 week
Sponsored by: Isilon Systems, Inc.
Tested by: kkenn

Slightly reorganize to reduce duplicated code.

Reviewed by: rwatson

When performing a VOP_LOOKUP() as part of UFS1 extended attribute
auto-start, set cnp.cn_lkflags to LK_EXCLUSIVE. This flag must now
be set so that lockmgr knows what kind of lock to acquire, and it
will panic if not specified. This resulted in a panic when using
extended attributes on UFS1 as of locking work present in the 6.x
branch.

This is a RELENG_6_0 merge candidate.

Reported by: lofi
MFC after: 3 days

- Fix botched LK_NOWAIT removal. I mistakenly thought this compiled as
part of GENERIC.

- LK_NOPAUSE is a nop now.

Sponsored by: Isilon Systems, Inc.

- UFS no longer uses PDIRUNLOCK to track the parent state. Instead, we now
rely on ufs to always leave the parent locked except in the ISDOTDOT
case. Adjust asserts to deal with these changes.

Sponsored by: Isilon Systems, Inc.

- Call VFS_ROOT() with LK_EXCLUSIVE.

Sponsored by: Isilon Systems, Inc.

Don't use VOP_LEASE() with operations on extended attribute backing
files.

Pointed out by: phk

Create a vnode object when the file is opened. Trust that we did so.

Ditch vfs_object_create() and make the callers call VOP_CREATEVOBJECT()
directly.

LINT defines things which compile in code that as referring to the old
a_desc element. change this to the new a_gen.a_desc to reflect
changes to vnode_if.h generation.

Noticed by: tinderbox, phk

Make VFS_ROOT() and vflush() take a thread argument.
This is to allow filesystems to decide based on the passed thread
which vnode to return.
Several filesystems used curthread, they now use the passed thread.

Annotate that we don't check the returned data length from ufs_readdir()
because UFS uses fixed-size directory blocks. When using this code with
other file systems, such as HFS+, the value of auio.uio_resid will need
to be taken into account.

Remove unnecessary setting of VV_SYSTEM on extended attribute backing
files. When this flag is used in our port of this code to Darwin, it
caused remarkable pain, and doesn't offer a benefit in FreeBSD.

Protect a non-text comment with a '-'.

White space cleanup: use spaces instead of tabs in variable declarations
local to a function. Remove a couple of blank lines in variable
declarations.

In one case, explicitly test against NULL rather than using a pointer
as a boolean directly.

Update my personal copyrights and NETA copyrights in the kernel
to use the "year1-year3" format, as opposed to "year1, year2, year3".
This seems to make lawyers more happy, but also prevents the
lines from getting excessively long as the years start to add up.

Suggested by: imp

Rename VOP_RMEXTATTR() to VOP_DELETEEXTATTR() for consistency with the
kernel ACL interfaces and system call names.

Break out UFS2 and FFS extattr delete and list vnode operations from
setextattr and getextattr to deleteextattr and listextattr, which
cleans up the implementations, and makes the results more readable,
and makes the APIs more clear.

Obtained from: TrustedBSD Project
Sponsored by: DARPA, Network Associates Laboratories

Add a "int fd" argument to VOP_OPEN() which in the future will
contain the filedescriptor number on opens from userland.

The index is used rather than a "struct file *" since it conveys a bit
more information, which may be useful to in particular fdescfs and /dev/fd/*

For now pass -1 all over the place.

Use __FBSDID().

Don't special-case handling of the empty string in the UFS1
extended attribute retrieval code: it's no longer special-cased,
and is caught by the normal UFS1 EA validity checks (and, in
fact, returns the same error, EINVAL).

Obtained from: TrustedBSD Project
Sponsored by: DARPA, Network Associates Laboratories

Jeff added locking assertions that the VV_ flags on vnodes were modified
only while holding appropriate vnode locks. This patch slides the lock
release for ufs_extattr_enable() to continue to hold the active vnode lock
on a backing file until after the flag change; it also acquires a vnode
lock when disabling an attribute and hence clearing a flag on the backing
vnode. This permits VFS_DEBUG_LOCKS to run UFS1 extended attributes
without panicking, as well as preventing a potential race and vnode flag
problem.

Approved by: re (jhb)
Pointed out by: DEBUG_VFS_LOCKS

Including <sys/stdint.h> is (almost?) universally only to be able to use
%j in printfs, so put a newsted include in <sys/systm.h> where the printf
prototype lives and save everybody else the trouble.

Back out M_* changes, per decision of the TRB.

Approved by: trb

Catch more uses of MIN().

Remove M_TRYWAIT/M_WAITOK/M_WAIT. Callers should use 0.
Merge M_NOWAIT/M_DONTWAIT into a single flag M_NOWAIT.

Print daddr_t's with %j and intmax_t.

Update licenses and wording: NAI has authorized the removal of clause three
of their BSD-style license; also, carry out the NAI Labs -> Network
Associates Laboratories renaming in these files.

size_t is not a struct (fix mislabelling in a comment).

Use canonical extattr_check_cred() instead of private implementation of the
same policy.

Sponsored by: DARPA & NAI Labs.

Fix credentials check: do not leak ENOATTR until we know if they're
supposed to know.

Sponsored by: DARPA & NAI Labs.

- Replace v_flag with v_iflag and v_vflag
- v_vflag is protected by the vnode lock and is used when synchronization
with VOP calls is needed.
- v_iflag is protected by interlock and is used for dealing with vnode
management issues. These flags include X/O LOCK, FREE, DOOMED, etc.
- All accesses to v_iflag and v_vflag have either been locked or marked with
mp_fixme's.
- Many ASSERT_VOP_LOCKED calls have been added where the locking was not
clear.
- Many functions in vfs_subr.c were restructured to provide for stronger
locking.

Idea stolen from: BSD/OS

This commit adds basic support for the UFS2 filesystem. The UFS2
filesystem expands the inode to 256 bytes to make space for 64-bit
block pointers. It also adds a file-creation time field, an ability
to use jumbo blocks per inode to allow extent like pointer density,
and space for extended attributes (up to twice the filesystem block
size worth of attributes, e.g., on a 16K filesystem, there is space
for 32K of attributes). UFS2 fully supports and runs existing UFS1
filesystems. New filesystems built using newfs can be built in either
UFS1 or UFS2 format using the -O option. In this commit UFS1 is
the default format, so if you want to build UFS2 format filesystems,
you must specify -O 2. This default will be changed to UFS2 when
UFS2 proves itself to be stable. In this commit the boot code for
reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c)
as there is insufficient space in the boot block. Once the size of the
boot block is increased, this code can be defined.

Things to note: the definition of SBSIZE has changed to SBLOCKSIZE.
The header file <ufs/ufs/dinode.h> must be included before
<ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and
ufs_lbn_t.

Still TODO:
Verify that the first level bootstraps work for all the architectures.
Convert the utility ffsinfo to understand UFS2 and test growfs.
Add support for the extended attribute storage. Update soft updates
to ensure integrity of extended attribute storage. Switch the
current extended attribute interfaces to use the extended attribute
storage. Add the extent like functionality (framework is there,
but is currently never used).

Sponsored by: DARPA & NAI Labs.
Reviewed by: Poul-Henning Kamp <phk@freebsd.org>

More s/file system/filesystem/g

Name ufs_vop_[gs]etextattr() consistently with the rest of our VOPs and
put then in the ufs_vnops where they belong, rather than in the ffs_vnops.

Ok'ed by: rwatson
Sponsored by: DARPA & NAI Labs.

Change the suser() API to take advantage of td_ucred as well as do a
general cleanup of the API. The entire API now consists of two functions
similar to the pre-KSE API. The suser() function takes a thread pointer
as its only argument. The td_ucred member of this thread must be valid
so the only valid thread pointers are curthread and a few kernel threads
such as thread0. The suser_cred() function takes a pointer to a struct
ucred as its first argument and an integer flag as its second argument.
The flag is currently only used for the PRISON_ROOT flag.

Discussed on: smp@

Remove references to vm_zone.h and switch over to the new uma API.

Update DBA for NAI. We have several. We used the wrong one. :-)

Add new errno ``ENOATTR''.

Simple p_ucred -> td_ucred changes to start using the per-thread ucred
reference.

Minor style tweaks.

Remove an unneeded comment and commented out code that won't be
needed.

Copyright + license update.

Part I: Update extended attribute API and ABI:

o Modify the system call syntax for extattr_{get,set}_{fd,file}() so
as not to use the scatter gather API (which appeared not to be used
by any consumers, and be less portable), rather, accepts 'data'
and 'nbytes' in the style of other simple read/write interfaces.
This changes the API and ABI.

o Modify system call semantics so that extattr_get_{fd,file}() return
a size_t. When performing a read, the number of bytes read will
be returned, unless the data pointer is NULL, in which case the
number of bytes of data are returned. This changes the API only.

o Modify the VOP_GETEXTATTR() vnode operation to accept a *size_t
argument so as to return the size, if desirable. If set to NULL,
the size will not be returned.

o Update various filesystems (pseodofs, ufs) to DTRT.

These changes should make extended attributes more useful and more
portable. More commits to rebuild the system call files, as well
as update userland utilities to follow.

Obtained from: TrustedBSD Project
Sponsored by: DARPA, NAI Labs

o Update copyright dates.
o Add reference to TrustedBSD Project in license header.
o Update dated comments, including comment in extattr.h claiming that
no file systems support extended attributes.
o Improve comment consistency.

o Clarify a comment about the locking condition of the vnode upon exit
from ufs_extattr_enable_with_open().
o Print auto-start notifications if (bootverbose). This was previously
commented out since it didn't know how to check for bootverbose.
o Drop in comments throughout indicating where ENOENT should be replaced
with ENOATTR once that is available.

Obtained from: TrustedBSD Project

o The comment about ordering the destruction of the lock and the removal of
the flag indicating that the structure was initialized didn't need
an XXX, since it didn't need fixing.

Obtained from: TrustedBSD Project

o Wrap a number of long lines of code, many of which were introduced
due to KSE-related (p) expansions.

Obtained from: TrustedBSD Project

Since namespace support was added to the UFS extended attribute
implementation to replace single-character namespace prefixes, '$' is no
longer an invalid attribute name, and the namespace is relevant to
validity determination.

o Remove '$' case from ufs_extattr_valid_attrname()
o Add attrnamespace argument to ufs_extattr_valid_attrname(), and
fill out appropriately.

Currently no decisions are made based on the namespace argument, but
may be in the future.

Obtained from: TrustedBSD Project

Change the kernel's ucred API as follows:
- crhold() returns a reference to the ucred whose refcount it bumps.
- crcopy() now simply copies the credentials from one credential to
another and has no return value.
- a new crshared() primitive is added which returns true if a ucred's
refcount is > 1 and false (0) otherwise.

KSE Milestone 2
Note ALL MODULES MUST BE RECOMPILED
make the kernel aware that there are smaller units of scheduling than the
process. (but only allow one thread per process at this time).
This is functionally equivalent to teh previousl -current except
that there is a thread associated with each process.

Sorry john! (your next MFC will be a doosie!)

Reviewed by: peter@freebsd.org, dillon@freebsd.org

X-MFC after: ha ha ha ha

Call vn_close on the backing file vnode if ufs_extattr_enable failed to
avoid leaking it.

Reviewed by: rwatson

Add a check to determine whether extended attributes have been
initialized on the file system before trying to grab the lock of the
per-mount extattr structure, as this lock is unitialized in that case.
This is needed because ufs_extattr_vnode_inactive is called from
ufs_inactive, which is also used by EA-unaware file systems such as
ext2fs.

Reviewed by: rwatson

o Merge contents of struct pcred into struct ucred. Specifically, add the
real uid, saved uid, real gid, and saved gid to ucred, as well as the
pcred->pc_uidinfo, which was associated with the real uid, only rename
it to cr_ruidinfo so as not to conflict with cr_uidinfo, which
corresponds to the effective uid.
o Remove p_cred from struct proc; add p_ucred to struct proc, replacing
original macro that pointed.
p->p_ucred to p->p_cred->pc_ucred.
o Universally update code so that it makes use of ucred instead of pcred,
p->p_ucred instead of p->p_pcred, cr_ruidinfo instead of p_uidinfo,
cr_{r,sv}{u,g}id instead of p_*, etc.
o Remove pcred0 and its initialization from init_main.c; initialize
cr_ruidinfo there.
o Restruction many credential modification chunks to always crdup while
we figure out locking and optimizations; generally speaking, this
means moving to a structure like this:
newcred = crdup(oldcred);
...
p->p_ucred = newcred;
crfree(oldcred);
It's not race-free, but better than nothing. There are also races
in sys_process.c, all inter-process authorization, fork, exec, and
exit.
o Remove sigio->sio_ruid since sigio->sio_ucred now contains the ruid;
remove comments indicating that the old arrangement was a problem.
o Restructure exec1() a little to use newcred/oldcred arrangement, and
use improved uid management primitives.
o Clean up exit1() so as to do less work in credential cleanup due to
pcred removal.
o Clean up fork1() so as to do less work in credential cleanup and
allocation.
o Clean up ktrcanset() to take into account changes, and move to using
suser_xxx() instead of performing a direct uid==0 comparision.
o Improve commenting in various kern_prot.c credential modification
calls to better document current behavior. In a couple of places,
current behavior is a little questionable and we need to check
POSIX.1 to make sure it's "right". More commenting work still
remains to be done.
o Update credential management calls, such as crfree(), to take into
account new ruidinfo reference.
o Modify or add the following uid and gid helper routines:
change_euid()
change_egid()
change_ruid()
change_rgid()
change_svuid()
change_svgid()
In each case, the call now acts on a credential not a process, and as
such no longer requires more complicated process locking/etc. They
now assume the caller will do any necessary allocation of an
exclusive credential reference. Each is commented to document its
reference requirements.
o CANSIGIO() is simplified to require only credentials, not processes
and pcreds.
o Remove lots of (p_pcred==NULL) checks.
o Add an XXX to authorization code in nfs_lock.c, since it's
questionable, and needs to be considered carefully.
o Simplify posix4 authorization code to require only credentials, not
processes and pcreds. Note that this authorization, as well as
CANSIGIO(), needs to be updated to use the p_cansignal() and
p_cansched() centralized authorization routines, as they currently
do not take into account some desirable restrictions that are handled
by the centralized routines, as well as being inconsistent with other
similar authorization instances.
o Update libkvm to take these changes into account.

Obtained from: TrustedBSD Project
Reviewed by: green, bde, jhb, freebsd-arch, freebsd-audit

Revert consequences of changes to mount.h, part 2.

Requested by: bde

Correct #includes to work with fixed sys/mount.h.

o Change the default from using IO_SYNC on EA set and delete operations
to not using IO_SYNC. Expose a sysctl (debug.ufs_extattr_sync) for
enabling the use of IO_SYNC.

- Use of IO_SYNC substantially degrades ACL performance when a
default ACL is set on a directory, as there are four synchronous
writes initiated to define both supporting EAs for new
sub-directories, and to set the data; two for new files. Later, this
may be optimized to two writes for sub-directories, one for new
files.

- IO_SYNC does not substantially improve consistency properties due
to the poor consistency properties of existing permissions (which
ACLs are a superset of), due to interaction with soft updates,
and due to differences in handling consistency for data and file
system meta-data.

- In macro-benchmarks, this reduces the overhead of setting default
ACLs down to the same overhead as enabling ACLs on a file system
and not using them. Enabling ACLs still introduces a small
overhead (I measure 7% on a -j 2 buildworld with pre-allocated
EA backing store, but this is not rigorous testing, nor in any way
optimized).

- The sysctl will probably change to another administration method
(or at least, a better name) in the near future, but consistency
properties of EAs are still being worked out. The toggle is defined
right now to allow easier performance analysis and exploration
of possible guarantees.

Obtained from: TrustedBSD Project

o Remove an unnecessary debugging printf from ufs_extattr_lookup(),
which resulted in the output of warning messages at boot if
UFS_EXTATTR_AUTOSTART was enabled but ".attribute" and possible
sub-directories weren't in a mounted MFS or UFS file systems.

Pointed out by: dcs
Obtained from: TrustedBSD Project

o Rename "namespace" argument to "attrnamespace" as namespace is a C++
reserved word.

Submitted by: jkh
Obtained from: TrustedBSD Project

o Change options FFS_EXTATTR and options FFS_EXTATTR_AUTOSTART to
options UFS_EXTATTR and UFS_EXTATTR_AUTOSTART respectively. This change
reflects the fact that our EA support is implemented entirely at the
UFS layer (modulo FFS start/stop/autostart hooks for mount and unmount
events). This also better reflects the fact that [shortly] MFS will also
support EAs, as well as possibly IFS.

o Consumers of the EA support in FFS are reminded that as a result, they
must change kernel config files to reflect the new option names.

Obtained from: TrustedBSD Project

o Caused FFS_EXTATTR_AUTOSTART to scan two sub-directories of ".attribute"
off of the file system root: "user" for user attributes, and "system"
for system attributes. When the scan occurs, attribute backing files
discovered in those directories will be started in the respective
namespaces. This re-introduces support for auto-starting of user
attributes, which was removed when the "$" prefix for system attributes
was replaced with explicit namespacing.

For users of the TrustedBSD UFS POSIX.1e ACL code, you'll need to:
mv ${FSROOT}/'$posix1e.acl_access' ${FSROOT}/system/posix1e.acl_access
mv ${FSROOT}/'$posix1e.acl_default' ${FSROOT}/system/posix1e.acl_default

For users of the TrustedBSD POSIX.1e Capability code, you'll need to:
mv ${FSROOT}/'$posix1e.cap' ${FSROOT}/system/posix1e.cap

For users of the TrustedBSD MAC code, you'll need to:
mv ${FSROOT}/'$freebsd.mac' ${FSROOT}/system/freebsd.mac

Updated versions of relevant patches will be released in the near
future.

Obtained from: TrustedBSD Project

o Change the API and ABI of the Extended Attribute kernel interfaces to
introduce a new argument, "namespace", rather than relying on a first-
character namespace indicator. This is in line with more recent
thinking on EA interfaces on various mailing lists, including the
posix1e, Linux acl-devel, and trustedbsd-discuss forums. Two namespaces
are defined by default, EXTATTR_NAMESPACE_SYSTEM and
EXTATTR_NAMESPACE_USER, where the primary distinction lies in the
access control model: user EAs are accessible based on the normal
MAC and DAC file/directory protections, and system attributes are
limited to kernel-originated or appropriately privileged userland
requests.

o These API changes occur at several levels: the namespace argument is
introduced in the extattr_{get,set}_file() system call interfaces,
at the vnode operation level in the vop_{get,set}extattr() interfaces,
and in the UFS extended attribute implementation. Changes are also
introduced in the VFS extattrctl() interface (system call, VFS,
and UFS implementation), where the arguments are modified to include
a namespace field, as well as modified to advoid direct access to
userspace variables from below the VFS layer (in the style of recent
changes to mount by adrian@FreeBSD.org). This required some cleanup
and bug fixing regarding VFS locks and the VFS interface, as a vnode
pointer may now be optionally submitted to the VFS_EXTATTRCTL()
call. Updated documentation for the VFS interface will be committed
shortly.

o In the near future, the auto-starting feature will be updated to
search two sub-directories to the ".attribute" directory in appropriate
file systems: "user" and "system" to locate attributes intended for
those namespaces, as the single filename is no longer sufficient
to indicate what namespace the attribute is intended for. Until this
is committed, all attributes auto-started by UFS will be placed in
the EXTATTR_NAMESPACE_SYSTEM namespace.

o The default POSIX.1e attribute names for ACLs and Capabilities have
been updated to no longer include the '$' in their filename. As such,
if you're using these features, you'll need to rename the attribute
backing files to the same names without '$' symbols in front.

o Note that these changes will require changes in userland, which will
be committed shortly. These include modifications to the extended
attribute utilities, as well as to libutil for new namespace
string conversion routines. Once the matching userland changes are
committed, a buildworld is recommended to update all the necessary
include files and verify that the kernel and userland environments
are in sync. Note: If you do not use extended attributes (most people
won't), upgrading is not imperative although since the system call
API has changed, the new userland extended attribute code will no longer
compile with old include files.

o Couple of minor cleanups while I'm there: make more code compilation
conditional on FFS_EXTATTR, which should recover a bit of space on
kernels running without EA's, as well as update copyright dates.

Obtained from: TrustedBSD Project

o Implement "options FFS_EXTATTR_AUTOSTART", which depends on
"options FFS_EXTATTR". When extended attribute auto-starting
is enabled, FFS will scan the .attribute directory off of the
root of each file system, as it is mounted. If .attribute
exists, EA support will be started for the file system. If
there are files in the directory, FFS will attempt to start
them as attribute backing files for attributes baring the same
name. All attributes are started before access to the file
system is permitted, so this permits race-free enabling of
attributes. For attributes backing support for security
features, such as ACLs, MAC, Capabilities, this is vital, as
it prevents the file system attributes from getting out of
sync as a result of file system operations between mount-time
and the enabling of the extended attribute. The userland
extattrctl tool will still function exactly as previously.
Files must be placed directly in .attribute, which must be
directly off of the file system root: symbolic links are
not permitted. FFS_EXTATTR will continue to be able
to function without FFS_EXTATTR_AUTOSTART for sites that do not
want/require auto-starting. If you're using the UFS_ACL code
available from www.TrustedBSD.org, using FFS_EXTATTR_AUTOSTART
is recommended.

o This support is implemented by adding an invocation of
ufs_extattr_autostart() to ffs_mountfs(). In addition,
several new supporting calls are introduced in
ufs_extattr.c:

ufs_extattr_autostart(): start EAs on the specified mount
ufs_extattr_lookup(): given a directory and filename,
return the vnode for the file.
ufs_extattr_enable_with_open(): invoke ufs_extattr_enable()
after doing the equililent of vn_open()
on the passed file.
ufs_extattr_iterate_directory(): iterate over a directory,
invoking ufs_extattr_lookup() and
ufs_extattr_enable_with_open() on each
entry.

o This feature is not widely tested, and therefore may contain
bugs, caution is advised. Several changes are in the pipeline
for this feature, including breaking out of EA namespaces into
subdirectories of .attribute (this is waiting on the updated
EA API), as well as a per-filesystem flag indicating whether
or not EAs should be auto-started. This is required because
administrators may not want .attribute auto-started on all
file systems, especially if non-administrators have write access
to the root of a file system.

Obtained from: TrustedBSD Project

Another round of the <sys/queue.h> FOREACH transmogriffer.

Created with: sed(1)
Reviewed by: md5(1)

Mechanical change to use <sys/queue.h> macro API instead of
fondling implementation details.

Created with: sed(1)
Reviewed by: md5(1)

o Commit reems of style(9) changes, whitespace improvements, and comment
cleanups.

Obtained from: TrustedBSD Project

o Zero the ufs_extattr_header length field (not necessary, but not a bad
idea either) in ufs_extattr_rm.
o More completely fill out the local_aio structure when writing out the
zero'd extended attribute in ufs_extattr_rm -- previoulsy, this worked
fine, but probably should not have. This corrects extraneous warnings
about inconsistent inodes following file deletion.

Reviewed by: jedgar

o Add an additional EA inconsistency reporting opportunity in
ufs_extattr_rm.
o Make both reporting locations report the function name where the
inconsistency is discovered, as well as the inode number in question.

Reviewed by: jedgar

o Make call to ufs_extattr_rm() in ufs_extattr_vnode_inactive() use
NULL as the credential, not 0, so as to make it more clear what's
going on.

Obtained from: TrustedBSD Project

o Remove unnecessary sanity check involving requested offset of extended
attribute read--the offset is required to be 0 by an earlier check,
meaning that it will always be within the scope of the attribute data.
This change should have no impact on executed code paths other than
removing the unnecessary check: please report if any new failures
start to occur as a result.

Obtained from: TrustedBSD Project

o Sanity check was inverted, resulting in a possible spurious panic
during unmount if extended attributes were in use. Correct by removing
an unneeded (and undesirable) '!'.

o Correct use of lockdestroy() by adding a new ufs_extattr_uepm_destroy()
call, which should be the last thing down to a per-mount extattr
management structure, after ufs_extattr_stop() on the file system.
This currently has the effect only of destroying the per-mount lock
on extended attributes, and clearing appropriate flags.
o Remove inappropriate invocation in ufs_extattr_vnode_inactive().

Convert lockmgr locks from using simple locks to using mutexes.

Add lockdestroy() and appropriate invocations, which corresponds to
lockinit() and must be called to clean up after a lockmgr lock is no
longer needed.

o Disallow privileged processes in jail() from directly accessing
system namespace extended attributes.
o Document privilege/jail() interaction relating to extended
attributes.

Obtained from: TrustedBSD Project

o Variety of extended attribute fixes
- In ufs_extattr_enable(), return EEXIST instead of EOPNOTSUPP
if the caller tries to configure an attribute name that is
already configured
- Throughout, add IO_NODELOCKED to VOP_{READ,WRITE} calls to
indicate lock status of passed vnode. Apparently not a
problem, but worth fixing.
- For all writes, make use of IO_SYNC consistent. Really,
IO_UNIT and combining of VOP_WRITE's should happen, but I
don't have that tested. At least with this, it's
consistent usage. (pointed out by: bde)
- In ufs_extattr_get(), fixed nested locking of backing
vnode (fine due to recursive lock support, but make it
more consistent with other code)
- In ufs_extattr_get(), clean up return code to set uio_resid
more consistently with other pieces of code (worked fine,
this is just a cleanup)
- Fix ufs_extattr_rm(), which was broken--effectively a nop.
- Minor comment and whitespace fixes.

Obtained from: TrustedBSD Project

Fix a 64-bitism. Use size_t instead of int for 4th argument to copyinstr.

Approved by: rwatson

Modify extended attribute protection model to authorize based on
attribute namespace and DAC protection on file:
- Attribute names beginning with '$' are in the system namespace
- The attribute name "$" is reserved
- System namespace attributes may only be read/set by suser()
or by kernel (cred == NULL)
- Other attribute names are in the application namespace
- The attribute name "" is reserved
- Application namespace attributes are protected in the manner
of the target file permission

o Kernel changes
- Add ufs_extattr_valid_attrname() to check whether the requested
attribute "set" or "enable" is appropriate (i.e., non-reserved)
- Modify ufs_extattr_credcheck() to accept target file vnode, not
to take inode uid
- Modify ufs_extattr_credcheck() to check namespace, then enforce
either kernel/suser for system namespace, or vaccess() for
application namespace
o EA backing file format changes
- Remove permission fields from extended attribute backing file
header
- Bump extended attribute backing file header version to 3
o Update extattrctl.c and extattrctl.8
- Remove now deprecated -r and -w arguments to initattr, as
permissions are now implicit
- (unrelated) fix error reporting and unlinking during failed
initattr to remove duplicate/inaccurate error messages, and to
only unlink if the failure wasn't in the backing file open()

Obtained from: TrustedBSD Project

o Correct spelling of ufs_exttatr_find_attr -> ufs_extattr_find_attr
o Add "const" qualifier to attrname argument of various calls to remove
warnings

Obtained from: TrustedBSD Project

Move the truncation code out of vn_open and into the open system call
after the acquisition of any advisory locks. This fix corrects a case
in which a process tries to open a file with a non-blocking exclusive
lock. Even if it fails to get the lock it would still truncate the
file even though its open failed. With this change, the truncation
is done only after the lock is successfully acquired.

Obtained from: BSD/OS

o Remove unneeded off_t variable to clean up compile warning

Obtained from: TrustedBSD Project

Don't allow VOP_GETEXTATTR to set uio->uio_offset != 0, as we don't
provide locking over extended attribute operations, requiring that
individual operations be atomic. Allowing non-zero starting offsets
permits applications/etc to put themselves at risk for inconsistent
behavior. As VOP_SETEXTATTR already prohibited non-zero write offsets,
this makes sense.

Suggested by: Andreas Gruenbacher <a.gruenbacher@bestbits.at>

o Introduce an extended attribute backing file header magic number
o Introduce an extended attribute backing file header version number

o Cause attribute data writes to use IO_SYNC since this improves the
chances of consistency with other file/directory meta-data in a
write. In the current set of extended attribute applications,
this does not hurt much. This should be discussed again later when
it comes time to optimize performance of attributes.

o Include an inode generation number in the per-attribute header
information. This allows consistency verification to catch when
a crash occurs, or an inode is recycled while attributes are not
properly configured. For now, an irritating error message is
displayed when an inconsistency occurs. At some point, may introduce
an ``extattrctl check ...'' which catches these before attributes
are enabled. Not today. If you get this message, it means you
somehow managed to get your attribute backing file out of synch
with the file system. When this occurs, attribute not found is
returned (== undefined). Writes will overwrite the value there
correcting the problem. Might want to think about introducing
a new errno or two to handle this kind of situation.

Discussed with: kris

Fix two bugs in extended attribute support for UFS/FFS:

o Put back in {} removed during over-zealous cleanup of gratuitous
debugging output during preparation for the commit. Due to the
missing {}, writes on extended attributes always silently failed.
Doh.

o Don't unlock the target vnode if it's the backing vnode, as we
don't lock the target vnode if it's the backing vnode.

Introduce extended attribute support for FFS, allowing arbitrary
(name, value) pairs to be associated with inodes. This support is
used for ACLs, MAC labels, and Capabilities in the TrustedBSD
security extensions, which are currently under development.

In this implementation, attributes are backed to data vnodes in the
style of the quota support in FFS. Support for FFS extended
attributes may be enabled using the FFS_EXTATTR kernel option
(disabled by default). Userland utilities and man pages will be
committed in the next batch. VFS interfaces and man pages have
been in the repo since 4.0-RELEASE and are unchanged.

o ufs/ufs/extattr.h: UFS-specific extattr defines
o ufs/ufs/ufs_extattr.c: bulk of support routines
o ufs/{ufs,ffs,mfs}/*.[ch]: hooks and extattr.h includes
o contrib/softupdates/ffs_softdep.c: extattr.h includes
o conf/options, conf/files, i386/conf/LINT: added FFS_EXTATTR

o coda/coda_vfsops.c: XXX required extattr.h due to ufsmount.h
(This should not be the case, and will be fixed in a future commit)

Currently attributes are not supported in MFS. This will be fixed.

Reviewed by: adrian, bp, freebsd-fs, other unthanked souls
Obtained from: TrustedBSD Project