Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r262623, r262667:

ext2fs: use of tab vs spaces.

Consistently use a single tab after a #define as mentioned in style(9).
Use tabs instead of space for indenting.
Fix a typo: "hash_vesion".

No functional change.

MFC r260988, r261034, r261120, r261235:

ext2fs: Properly the EXT4_EXTENTS and EXT4_INDEX to the inode flags.

In order to support Ext4 extents we need to pass the Ext4 inode flags
without interfering with the chflags. This is better done by using the
i_flag field in the inode and doing proper translation to the linux
ext4 equivalents.

Solve a potential corruption issue in the dirindex code. The dirindex
code can now be renabled as the problems related to it have been
solved.

Suggested by: bde
Tested by: kevlo

MFC r255338:

ext2fs: temporarily disable htree directory index.

In addition to our implementation not having workarounds for hash
collisions, it appears we also have a compatibility problem.

For now disable the htree code until we are able to re-examine
both issues.

PR: kern/183230

MFC r253651, r253652, r253657;

ext2fs: Don't assume that on-disk format of a directory is the same
as in <sys/dirent.h>

ext2_readdir() has always been very fs specific and different
with respect to its ufs_ counterpart. Recent changes from UFS
have made it possible to share more closely the implementation.

Always start parsing at DIRBLKSIZ aligned offset, skip first entries if
uio_offset is not DIRBLKSIZ aligned. Return EINVAL if buffer is too
small for single entry.

Preallocate buffer for cookies.

Skip entries with zero inode number.

Reviewed by: gleb, Zheng Liu

MFC r252890, 252906, r252907, r253861, r254104:

Implementation of the HTree directory index.

This is a port of NetBSD's GSoC 2012 Ext3 HTree directory indexing
by Vyacheslav Matyushin. It was cleaned up and enhanced for FreeBSD
by Zheng Liu (lz@).

This is an excellent example of work shared among different projects:
Vyacheslav was able to look at an early prototype from Zheng Liu who
was also able to check the code from Haiku (with permission).

As in linux, the feature is not available by default and must be
enabled explicitly with tune2fs. We still do not support the
workarounds required in readdir for NFS.

Submitted by: Zheng Liu
Tested by: Mike Ma
Sponsored by: Google Inc.

MFC r252364:

Bring some updates from ufs/ufs_lookup to ext2fs.

Don't set IN_CHANGE and IN_UPDATE on inodes for potentially suspended
file systems.

Only set i_offset in the parent directory's i-node during a lookup for
non-LOOKUP operations.
Relax a VOP assertion for a DELETE lookup.

Move the code from ufs_lookup.c used to do dotdot lookup, into
the helper function. It is supposed to be useful for any filesystem
that has to unlock dvp to walk to the ".." entry in lookup routine.

MFC r251658, 251823:

Turn DIAGNOSTICs to INVARIANTS in ext2fs. following other filesystems.

MFC r246563, r246564, r246634:

ext2fs: Miscellaneous cleanups and fixes.

make e2fs_maxcontig local and remove tautological check.
Replace redundant EXT2_MIN_BLOCK with EXT2_MIN_BLOCK_SIZE.
Use prototype declarations for function definitions

Submitted by: Christoph Mallon

MFC r246347, r246348, r246349, r246350, r246351, r246352:

ext2fs: Miscellaneous cleanups and fixes.

Use EXT2_LINK_MAX instead of LINK_MAX.
Use nitems().
Correct off-by-one errors in FFTODT() and DDTOFT().
Remove useless rootino local variable.
Remove unused em_e2fsb definition.
Move assignment where it is not dead.

Submitted by: Christoph Mallon

MFC r240355, r239372, r246258;

ext2fs: general cleanups.

- Remove unused extern declarations in fs.h
- Correct comments in ext2_dir.h
- Several panic() messages showed wrong function names.
- Remove commented out stray line in ext2_alloc.c.
- Remove the unused macro EXT2_BLOCK_SIZE_BITS() and the then
write-only member e2fs_blocksize_bits from struct m_ext2fs.
- Remove the unused macro EXT2_FIRST_INO() and the then write-only
member e2fs_first_inode from struct m_ext2fs.
- Remove EXT2_DESC_PER_BLOCK() and the member e2fs_descpb from
struct m_ext2fs.
- Remove the unused members e2fs_bmask, e2fs_dbpg and
e2fs_mount_opt from struct m_ext2fs
- Correct harmless off-by-one error for fspath in ext2_vfsops.c.
- Remove the unused and broken macros EXT2_ADDR_PER_BLOCK_BITS()
and EXT2_DESC_PER_BLOCK_BITS().
- Remove the !_KERNEL versions of the EXT2_* macros.

Submitted by: Christoph Mallon

To ease the ease bringing this change I also brought this changes:
- Fix typo.
- Fix style nit.

MFC: r235508

Fix a couple of issues that appear to be inherited from the old
8.x code:
- If the lock cannot be acquired immediately unlocks 'bar' vnode
and then locks both vnodes in order.
- wrong vnode type panics from cache_enter_time after calls by
ext2_lookup.

The fix merges changes from ufs/ufs_lookup.c.

Submitted by: Mateusz Guzik
Approved by: jhb (mentor)

MFC r231949:
Fix found places where uio_resid is truncated to int.

Add the sysctl debug.iosize_max_clamp, enabled by default. Setting the
sysctl to zero allows to perform the SSIZE_MAX-sized i/o requests from
the usermode.

MFC r232493:
Remove unneeded cast to u_int. The values as small enough to fit into
int, beside the use of MIN macro which performs type promotions.

MFC r232494:
Instead of incomplete handling of read(2)/write(2) return values that
does not fit into registers, declare that we do not support this case
using CTASSERT(), and remove endianess-unsafe code to split return value
into td_retval.

While there, change the style of the sysctl debug.iosize_max_clamp
definition.

MFC r232495:
pipe_read(): change the type of size to int, and remove signed clamp.
pipe_write(): change the type of desiredsize back to int, its value fits.

MFC: r228507, r228539, r228583

Merge ext2_readwrite.c into ext2_vnops.c as done in UFS.
Bring in reallocblk to ext2fs: new feature implemented by Zheng Liu as
GSoC 2010.
Many style fixes by jh@.

PR: 159232, 159233 and 162564
Approved by: jhb (mentor)

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Merge comments about converting directory entries to be more direct and
concise.

Inspired by: Gleb Kurtsou

Sync with several changes in UFS/FFS:
- 77115: Implement support for O_DIRECT.
- 98425: Fix a performance issue introduced in 70131 that was causing
reads before writes even when writing full blocks.
- 98658: Rename the BALLOC flags from B_* to BA_* to avoid confusion with
the struct buf B_ flags.
- 100344: Merge the BA_ and IO_ flags so so that they may both be used in
the same flags word. This merger is possible by assigning the IO_ flags
to the low sixteen bits and the BA_ flags the high sixteen bits.
- 105422: Fix a file-rewrite performance case.
- 129545: Implement IO_INVAL in VOP_WRITE() by marking the buffer as
"no cache".
- Readd the DOINGASYNC() macro and use it to control asynchronous writes.
Change i-node updates to honor DOINGASYNC() instead of always being
synchronous.
- Use a PRIV_VFS_RETAINSUGID check instead of checking cr_uid against 0
directly when deciding whether or not to clear suid and sgid bits.

Submitted by: Pedro F. Giffuni giffunip at yahoo

Use a private EXT2_ROOTINO constant instead of redefining ROOTINO.

Submitted by: Pedro F. Giffuni giffunip at yahoo

Move calculation of 'bmask' earlier to match it's current location in
ufs_lookup().

Bring in the ext2fs work done by Aditya Sarawgi during and after Google Summer
of Code 2009:

- BSDL block and inode allocation policies for ext2fs. This involves the use
FFS1 style block and inode allocation for ext2fs. Preallocation was removed
since it was GPL'd.
- Make ext2fs MPSAFE by introducing locks to per-mount datastructures.
- Fixes for kern/122047 PR.
- Various small bugfixes.
- Move out of gnu/ directory.

Sponsored by: Google Inc.
Submitted by: Aditya Sarawgi <sarawgi.aditya AT SPAMFREE gmail DOT com>

Do not use casts (int *)0 and (struct thread *)0 for the arguments of
vn_rdwr, use NULL.

Reviewed by: jhb
MFC after: 1 week

- Sync our copies of ext2fs Linux headers to current Linux versions.
Minimize differencies between our ext2fs headers and relevant Linux
versions by using EXT2_SB macro to access the superblock fields. Most
of the differencies in access to these fields are now hidden inside
this macro.
- Rename the s_db_per_group field of ext2fs_sb_info to s_gdb_count
to reflect the similar change in Linux headers. New name also seem
to be more appropriate for this field.
- Use proper types for s_first_inode and s_inode_size in-core superblock
fields. Now they reflec types used in the on-disk superblock version.
- Add support for older filesystem revisions that doesn't have proper
s_first_ino and s_inode_size fields in the on-disk superblock. In these
cases predefined values for these fields are used.
- Add simple sanity checks for s_first_inode and s_inode_size correctness.

Reviewed by: bde (previous version)
MFC after: 2 weeks

Retire the MALLOC and FREE macros. They are an abomination unto style(9).

MFC after: 3 months

VOP_LOCK1() (and so VOP_LOCK()) and VOP_UNLOCK() are only used in
conjuction with 'thread' argument passing which is always curthread.
Remove the unuseful extra-argument and pass explicitly curthread to lower
layer functions, when necessary.

KPI results broken by this change, which should affect several ports, so
version bumping and manpage update will be further committed.

Tested by: kris, pho, Diego Sardina <siarodx at gmail dot com>

vn_lock() is currently only used with the 'curthread' passed as argument.
Remove this argument and pass curthread directly to underlying
VOP_LOCK1() VFS method. This modify makes the code cleaner and in
particular remove an annoying dependence helping next lockmgr() cleanup.
KPI results, obviously, changed.

Manpage and FreeBSD_version will be updated through further commits.

As a side note, would be valuable to say that next commits will address
a similar cleanup about VFS methods, in particular vop_lock1 and
vop_unlock.

Tested by: Diego Sardina <siarodx at gmail dot com>,
Andrea Di Pasquale <whyx dot it at gmail dot com>

Fix -Wundef warnings found when compiling i386 LINT, GENERIC and
custom kernels.

Apply the same fix to a potential race in the ISDOTDOT code in
ext2_lookup() that was used to fix an actual race in ufs_lookup.c:1.78.
This is not currently a hazard, but the bug would be activated by
marking ext2fs as MPSAFE.

Requested by: bde
MFC after: 2 weeks

Move ext2fs from src/gnu to src/gnu/fs.
Discussed on arch@.

Reviewed by: kan
Approved by: re (blanket), kan

- Change all filesystems and vfs_cache to relock the dvp once the child is
locked in the ISDOTDOT case. Se vfs_lookup.c r1.79 for details.

Sponsored by: Isilon Systems, Inc.

- Remove wantparent, it is no longer necessary. An assert in vfs_lookup.c
prevents any callers from doing a modifying op without
LOCKPARENT or WANTPARENT.

- ext2fs_lookup() is no longer responsible for unlocking the dvp, this is
handled in vfs_lookup.c. This code was missing PDIRUNLOCK use prior
to the removal of PDIRUNLOCK in rev 1.73 of vfs_lookup.c.

Sponsored by: Isilon Systems, Inc.

Make a SYSCTL_NODE static

/* -> /*- for copyright notices, minor format tweaks as necessary

Remove advertising clause from University of California Regent's
license, per letter dated July 22, 1999 and email from Peter Wemm,
Alan Cox and Robert Watson.

Approved by: core, peter, alc, rwatson

Properly vector all bwrite() and BUF_WRITE() calls through the same path
and s/BUF_WRITE()/bwrite()/ since it now does the same as bwrite().

Fixed misspellings of "ext2_*" as "ufs_*" and " "ext2fs_*", and of
"independent" as "dependent" Fixed some other relatively minor wording
and formatting errors.

Clean up whitespace, s/register //, refrain from strong urge to ANSIfy.

uiomove-related caddr_t -> void * (just the low-hanging fruit)

Back out M_* changes, per decision of the TRB.

Approved by: trb

Remove M_TRYWAIT/M_WAITOK/M_WAIT. Callers should use 0.
Merge M_NOWAIT/M_DONTWAIT into a single flag M_NOWAIT.

MFufs 1.33:
In the 'found' case for ext2_lookup() the underlying bp's data was
being accessed after the bp had been releaed. A simple move of the
brelse() solves the problem.

The PR reports that this caused panics running the GDB testsuite unless
NO_GEOM is configured.

PR: 44060
Reported by: Mark Kettenis <kettenis@chello.nl>
MFC after: 3 days

In order to better support flexible and extensible access control,
make a series of modifications to the credential arguments relating
to file read and write operations to cliarfy which credential is
used for what:

- Change fo_read() and fo_write() to accept "active_cred" instead of
"cred", and change the semantics of consumers of fo_read() and
fo_write() to pass the active credential of the thread requesting
an operation rather than the cached file cred. The cached file
cred is still available in fo_read() and fo_write() consumers
via fp->f_cred. These changes largely in sys_generic.c.

For each implementation of fo_read() and fo_write(), update cred
usage to reflect this change and maintain current semantics:

- badfo_readwrite() unchanged
- kqueue_read/write() unchanged
pipe_read/write() now authorize MAC using active_cred rather
than td->td_ucred
- soo_read/write() unchanged
- vn_read/write() now authorize MAC using active_cred but
VOP_READ/WRITE() with fp->f_cred

Modify vn_rdwr() to accept two credential arguments instead of a
single credential: active_cred and file_cred. Use active_cred
for MAC authorization, and select a credential for use in
VOP_READ/WRITE() based on whether file_cred is NULL or not. If
file_cred is provided, authorize the VOP using that cred,
otherwise the active credential, matching current semantics.

Modify current vn_rdwr() consumers to pass a file_cred if used
in the context of a struct file, and to always pass active_cred.
When vn_rdwr() is used without a file_cred, pass NOCRED.

These changes should maintain current semantics for read/write,
but avoid a redundant passing of fp->f_cred, as well as making
it more clear what the origin of each credential is in file
descriptor read/write operations.

Follow-up commits will make similar changes to other file descriptor
operations, and modify the MAC framework to pass both credentials
to MAC policy modules so they can implement either semantic for
revocation.

Obtained from: TrustedBSD Project
Sponsored by: DARPA, NAI Labs

Pass IO_NOMACCHECK to vn_rdwr() in the following checks to prevent
enforcement of MAC policy on the read or write operations:

- In ext2fs, don't enforce MAC on loop-back reads and writes supporting
directory read operations in lookup(), directory modifications in
rename(), directory write operations in mkdir(), symlink write
operations in symlink().

- In the NFS client locking code, perform vn_rdwr() on the NFS locking
socket without enforcing MAC, since the write is done on behalf of
the kernel NFS implementation rather than the user process.

- In UFS, don't enforce MAC on loop-back reads and writes supporting
directory read operations in lookup(), and symlink write operations
in symlink().

Obtained from: TrustedBSD Project
Sponsored by: DARPA, NAI Labs

Give ext2fs its own static "dirchk" variable instead of using ufs's
variable. Make this accessible as the sysctl vfs.e2fs.dirchk.

Remove register keyword.

Complete the separation of ext2fs from ufs by copying the remaining
shared code and converting all ufs references. Originally it may
have made sense to share common features between the two filesystems,
but recently it has only caused problems, the UFS2 work being the
final straw.

All UFS_* indirect calls are now direct calls to ext2_* functions,
and ext2fs-specific mount and inode structures have been introduced.

Moved $FreeBSD$ to the correct place.

Fixed some style bugs in the removal of __P(()). Continuation lines
were not outdented to preserve non-KNF lining up of code with parentheses.
Switch to KNF formatting.

Remove __P.

Add a flags parameter to VFS_VGET to pass through the desired
locking flags when acquiring a vnode. The immediate purpose is
to allow polling lock requests (LK_NOWAIT) needed by soft updates
to avoid deadlock when enlisting other processes to help with
the background cleanup. For the future it will allow the use of
shared locks for read access to vnodes. This change touches a
lot of files as it affects most filesystems within the system.
It has been well tested on FFS, loopback, and CD-ROM filesystems.
only lightly on the others, so if you find a problem there, please
let me (mckusick@mckusick.com) know.

KSE Milestone 2
Note ALL MODULES MUST BE RECOMPILED
make the kernel aware that there are smaller units of scheduling than the
process. (but only allow one thread per process at this time).
This is functionally equivalent to teh previousl -current except
that there is a thread associated with each process.

Sorry john! (your next MFC will be a doosie!)

Reviewed by: peter@freebsd.org, dillon@freebsd.org

X-MFC after: ha ha ha ha

Convert all users of fldoff() to offsetof(). fldoff() is bad
because it only takes a struct tag which makes it impossible to
use unions, typedefs etc.

Define __offsetof() in <machine/ansi.h>

Define offsetof() in terms of __offsetof() in <stddef.h> and <sys/types.h>

Remove myriad of local offsetof() definitions.

Remove includes of <stddef.h> in kernel code.

NB: Kernelcode should *never* include from /usr/include !

Make <sys/queue.h> include <machine/ansi.h> to avoid polluting the API.

Deprecate <struct.h> with a warning. The warning turns into an error on
01-12-2000 and the file gets removed entirely on 01-01-2001.

Paritials reviews by: various.
Significant brucifications by: bde

Fixed some serious bugs in ext2_readdir():

The cookie buffer was usually overrun by a large amount whenever
cookies were used. Cookies are used by nfs and the Linuxulator, so
this bug usually caused panics whenever an ext2fs filesystem was nfs
mounted or a Linux utility that calls readdir() was run on an ext2fs
filesystem.

The directory buffer was sometimes overrun by a small amount. This
sometimes caused panics and wrong results even for FreeBSD utilities,
but it was usually harmless because FreeBSD utilities use a large
enough buffer size (4K). Linux utilities usually triggered the bug
since they use a too-small buffer size (512 bytes), at least with the
old RedHat utilities that I tested with.

PR: 19407 (this fix is incomplete or for a slightly different bug)

Separate the struct bio related stuff out of <sys/buf.h> into
<sys/bio.h>.

<sys/bio.h> is now a prerequisite for <sys/buf.h> but it shall
not be made a nested include according to bdes teachings on the
subject of nested includes.

Diskdrivers and similar stuff below specfs::strategy() should no
longer need to include <sys/buf.> unless they need caching of data.

Still a few bogus uses of struct buf to track down.

Repocopy by: peter

ext2fs relies on UFS support code, and as a result also requires
extattr.h to be included. This fixes the broken ext2fs build as of
the import of extattr code.

Also added $FreeBSD: $ to a couple of files that didn't have them,
without which I couldn't commit this fix.

Reported by: "George W. Dinolt" <gdinolt@pacbell.net>

Rename the existing BUF_STRATEGY() to DEV_STRATEGY()

substitute BUF_WRITE(foo) for VOP_BWRITE(foo->b_vp, foo)

substitute BUF_STRATEGY(foo) for VOP_STRATEGY(foo->b_vp, foo)

This patch is machine generated except for the ccd.c and buf.h parts.

MFS (ext2_lookup.c 1.17.2.2, ext2_vnops.c 1.42.2.2: fix "filetype" support).

Approved by: jkh

Support filesystems with the not-so-new "filetype" feature. This
feature gives the d_type field for struct dirent. We used to panic
in ext2_readdir() for filesystems with this feature.

Add a vnode argument to VOP_BWRITE to get rid of the last vnode
operator special case. Delete special case code from vnode_if.sh,
vnode_if.src, umap_vnops.c, and null_vnops.c.

Fix warnings in preparation for adding -Wall -Wcast-qual to the
kernel compile

Fixed printf format errors.

Back out DIAGNOSTIC changes.

Turn DIAGNOSTIC into a new-style option.

Removed __FreeBSD__ ifdefs.

VFS mega cleanup commit (x/N)

1. Add new file "sys/kern/vfs_default.c" where default actions for
VOPs go. Implement proper defaults for ABORTOP, BWRITE, LEASE,
POLL, REVOKE and STRATEGY. Various stuff spread over the entire
tree belongs here.

2. Change VOP_BLKATOFF to a normal function in cd9660.

3. Kill VOP_BLKATOFF, VOP_TRUNCATE, VOP_VFREE, VOP_VALLOC. These
are private interface functions between UFS and the underlying
storage manager layer (FFS/LFS/MFS/EXT2FS). The functions now
live in struct ufsmount instead.

4. Remove a kludge of VOP_ functions in all filesystems, that did
nothing but obscure the simplicity and break the expandability.
If a filesystem doesn't implement VOP_FOO, it shouldn't have an
entry for it in its vnops table. The system will try to DTRT
if it is not implemented. There are still some cruft left, but
the bulk of it is done.

5. Fix another VCALL in vfs_cache.c (thanks Bruce!)

Remove some stuff from lookup which is now handled centrally.

Uncut&paste cache_lookup().

This unifies several times in theory indentical 50 lines of code.

The filesystems have a new method: vop_cachedlookup, which is the
meat of the lookup, and use vfs_cache_lookup() for their vop_lookup
method. vfs_cache_lookup() will check the namecache and pass on
to the vop_cachedlookup method in case of a miss.

It's still the task of the individual filesystems to populate the
namecache with cache_enter().

Filesystems that do not use the namecache will just provide the
vop_lookup method as usual.

Removed unused #includes.

Support NFS cookies in VOP_READDIR, allowing ext2fs filesystems to be
exported via NFS.

2.2 candidate.

Fixed gratuitous ANSIisms.

Removed trailing newline from panic messages.

This is the kernel Lite/2 commit. There are some requisite userland
changes, so don't expect to be able to run the kernel as-is (very well)
without the appropriate Lite/2 userland changes.

The system boots and can mount UFS filesystems.

Untested: ext2fs, msdosfs, NFS
Known problems: Incorrect Berkeley ID strings in some files.
Mount_std mounts will not work until the getfsent
library routine is changed.

Reviewed by: various people
Submitted by: Jeffery Hsu <hsu@freebsd.org>

Fixed lookup of ".." in checkpath. It always failed, so renames of
directories to a different parent directory always failed. This bug
was caused by 4.4Lite2 changing the directory format and ext2fs not
keeping up.

Should be in 2.2.

Fixed spacefree calculation in ext2_direnter(). This bug sometimes caused
panics.

This should be in 2.2, of course.

Submitted by: davidg
Obtained from: bouyer@antioche.ibp.fr (Manuel BOUYER) (fix for NetBSD)

Clean up -Wunused warnings.

Reviewed by: bde

ext2_inode_cnv.c:
Included <sys/vnode.h> and its prerequisite <sys/proc.h>, and cleaned
up includes. The vop_t changes made the non-inclusion of vnode.h
fatal instead of just sloppy.

i386_bitops.h:
Changed `extern inline' to `static inline'. `extern inline' is a
Linuxism that stops things from compiling without -O. Fixed
idempotency identifier.

Misc:
Added prototypes. Staticized some functions so that prototypes are
unnecessary. Added casts. Cleaned up includes.

Cleaned up some lint and some obvious prototyping errors.

Main code for the ext2fs filesystem. Please refer to the COPYRIGHT.INFO
file for GPL restrictions. This code was ported to the BSD platform
by Godmar Back <gback@facility.cs.utah.edu> and specifically to FreeBSD
by John Dyson. This code is still green and should be used with caution.
Additional changes to UFS necessary to make this code work will be commited
seperately.
Submitted by: Godmar Back <gback@facility.cs.utah.edu>
Obtained from: Lites/Mach4