As FSTRANS is part of VOP_*LOCK() since June 4, 2017 the vdead_check()
from union_lock() is no longer needed.

Adapt union_lock() to the recent addition of upgrade or downgrade.

VV_LOCKSWORK now.

s/unaccessible/inaccessible/

Fix previous, don't copy up if the underlying node is unreadable.

In union_access() copy up regular files before checking permissions.

Unionfs is meant to provide a writable layer above a read-only layer
and should not fail here just because the lower layer is mounted
read-only.

Overhaul of the EVFILT_VNODE kevent(2) filter:

- Centralize vnode kevent handling in the VOP_*() wrappers, rather than
forcing each individual file system to deal with it (except VOP_RENAME(),
because VOP_RENAME() is a mess and we currently have 2 different ways
of handling it; at least it's reasonably well-centralized in the "new"
way).
- Add support for NOTE_OPEN, NOTE_CLOSE, NOTE_CLOSE_WRITE, and NOTE_READ,
compatible with the same events in FreeBSD.
- Track which kevent notifications clients are interested in receiving
to avoid doing work for events no one cares about (avoiding, e.g.
taking locks and traversing the klist to send a NOTE_WRITE when
someone is merely watching for a file to be deleted, for example).

In support of the above:

- Add support in vnode_if.sh for specifying PRE- and POST-op handlers,
to be invoked before and after vop_pre() and vop_post(), respectively.
Basic idea from FreeBSD, but implemented differently.
- Add support in vnode_if.sh for specifying CONTEXT fields in the
vop_*_args structures. These context fields are used to convey information
between the file system VOP function and the VOP wrapper, but do not
occupy an argument slot in the VOP_*() call itself. These context fields
are initialized and subsequently interpreted by PRE- and POST-op handlers.
- Version VOP_REMOVE(), uses the a context field for the file system to report
back the resulting link count of the target vnode. Return this in tmpfs,
udf, nfs, chfs, ext2fs, lfs, and ufs.

NetBSD 9.99.92.

Fix union_parsepath(), either the upper or the lower dvp may be NULL.

Now remove cn_consume from struct componentname.

This change requires a kernel bump.

Note though that I'm not going to version the VOP_LOOKUP args
structure (or any other args structure) as code that doesn't touch
cn_consume doesn't need attention and code that does will fail on it
without further intervention.

Onionfs needs to know about parsepath too, in case it has one of the
other cases underneath it.

The solution here is not really very good (take the longer
path-to-consume if they're different) but it will serve for the cases
that exist.

(If we were to add a fs that really uses different naming semantics,
we'd have to take additional steps; probably it doesn't make sense to
allow unionfs to union such a thing with a normal fs and attempting it
should fail at mount time.)

Update fs/unionfs as well to avoid increasing the current set of
compile failures there. Though maybe it's time to just remove
fs/unionfs.

- Add a new vnode op: VOP_PARSEPATH.
- Move namei_getcomponent to genfs_vnops.c and call it genfs_parsepath.
- Add a parsepath entry to every vnode ops table.

VOP_PARSEPATH takes a directory vnode to be searched and a complete
following path and chooses how much of that path to consume. To begin
with, all parsepath calls are genfs_parsepath, which locates the first
'/' as always.

Note that the call doesn't take the whole struct componentname, only
the string. The other bits of struct componentname should not be
needed and there's no reason to cause potential complications by
exposing them.

branches: 1.74.6;
Operation union_readdirhook() stores the lower directory as un_uppervp.
This breaks the assumption that un_uppervp->v_mount is the upper mount.

Fix by storing the directory as un_lowervp and adapt union_readdir().

Should fix PR kern/55552: panic with union mount

Add ACL support for FFS. From FreeBSD.

UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.70.10; 1.70.14; 1.70.16;
Make VOP_RECLAIM do the last unlock of the vnode.

VOP_RECLAIM naturally has exclusive access to the vnode, so having it
locked on entry is not strictly necessary -- but it means if there
are any final operations that must be done on the vnode, such as
ffs_update, requiring exclusive access to it, we can now kassert that
the vnode is locked in those operations.

We can't just have the caller release the last lock because some file
systems don't use genfs_lock, and require the vnode to remain valid
for VOP_UNLOCK to work, notably unionfs.

Use VCALL() to lock or unlock the lower node.

Move v_writecount adjustment from revoke to reclaim.

branches: 1.67.2;
Change VOP_REMOVE and VOP_RMDIR to preserve lock/ref on dvp.

No change to vp -- the plan is to replace the node by the
componentname in the vop parameters, and let all directory vops do
lookups internally.

Proposed on tech-kern with no objections:
https://mail-index.netbsd.org/tech-kern/2017/04/17/msg021825.html

Remove unused argument "nextp" from vfs_busy() and vfs_unbusy().
Remove argument "keepref" from vfs_unbusy() and add vfs_ref() where needed.

Make VOP_INACTIVE preserve vnode lock on return.

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2017/04/01/msg021751.html

Ride 7.99.68, a bumpy bus of incremental vfs improvements!

Handle v_writecount from union_open(), union_close() and union_revoke()
so lower file system vnodes get marked as open for writing.

branches: 1.63.2; 1.63.4;
Make VOP_LINK return directory still locked and referenced.

Ride 7.99.10 bump.

branches: 1.62.4;
Add VOP_FALLOCATE and VOP_FDISCARD to every vnode ops table I can
find.

The filesystem ones all call genfs_eopnotsupp - right now I am only
implementing the plumbing and we can implement fallocate and/or
fdiscard for files later.

The device ones call spec_fallocate (which is also genfs_eopnotsupp)
and spec_fdiscard, which dispatches to the device-level op.

The fifo ones all call vn_fifo_bypass, which also ends up being
EOPNOTSUPP.

Also set or assert that *vpp is null before calling VOP_MKDIR.

Set *vpp to NULL before calling VOP_CREATE. This always happens when
calling using nameidata, and if not something went wrong, so we'd like
to be able to assert about it.

branches: 1.59.2;
- Make VI_XLOCK, VI_CLEAN and VI_LOCKSHARE private to kern/vfs_*.c.
- Make vwait() static.
- Add vdead_check() to check a vnode for being or becoming dead.

Discussed on tech-kern.

Welcome to 6.99.38

Restructure union_lock() to always lock before testing for dead node.
Add two little helpers to lock or unlock a node. Use "vp" for the
union node and "lockvp" for the node to be locked. Use ISSET() to
test flags, add assertions.

The current implementation of vn_lock() is racy. Modification of
the vnode operations vector for active vnodes is unsafe because it
is not known whether deadfs or the original file system will be
called.

- Pass down LK_RETRY to the lock operation (hint for deadfs only).

- Change deadfs lock operation to return ENOENT if LK_RETRY is unset.

- Change all other lock operations to check for dead vnode once
the vnode is locked and unlock and return ENOENT in this case.

With these changes in place vnode lock operations will never succeed
after vclean() has marked the vnode as VI_XLOCK and before vclean()
has changed the operations vector.

Adresses PR kern/37706 (Forced unmount of file systems is unsafe)

Discussed on tech-kern.

Welcome to 6.99.33

Change union_allocvp() to take an unlocked uppervp and to return the
union node unlocked. Another VI_XLOCK hack is gone.

Remove an unused variable

Get rid of UN_KLOCK to keep a lock on vput(). It is not really needed
and makes the source difficult to read. Always hold references to the
union nodes until the operation is done.

Fix the DOT and DOTDOT case for union_lookup1().

Change vnode operation lookup to return the resulting vnode *vpp unlocked.
Change cache_lookup() to return an unlocked vnode.

Discussed on tech-kern@

Welcome to 6.99.31

Change vnode operations create, mknod, mkdir and symlink to return
the resulting vnode *vpp unlocked.

Discussed on tech-kern@

Welcome to 6.99.30

Change vnode operations create, mknod, mkdir and symlink to keep the
directory node dvp locked on return.

Discussed on tech-kern@

Welcome to 6.99.29

branches: 1.49.8; 1.49.12;
Replace flag based union node locking with generic vnode lock, support
shared and nowait locks and protect un_uppervp and un_*sz with mutex.

Mark file system MPSAFE.

VOP_ABORTOP() has no specific lock requirements so there is no need
to force locked vnodes here. It should be impossible to come here
with a nil upper node.

Relock the directory vnode after copyup. A locked union node with an
unlocked upper vnode can no longer exist so make FIXUP() an assertion.

branches: 1.47.2;
VOP_GETATTR() needs a shared lock at least.

Stop abusing relookup() to prepare the creation of new nodes
in the upper layer.
Replace union_relookup() with union_do_lookup() that prepares
a component, calls VOP_LOOKUP() and does the EEXIST test.

Change some `#ifdef DIAGNOSTIC' to `KASSERT'.
Instead of a `pid_t' use a `lwp_t *' for locking diagnostics.

No functional changes intended.

Add missing parts to mount devices from a union file system:
- union_close() has to lock/unlock the lower vnode.
- union_fsync() has to call spec_fsync() for the union vnode.
- union_strategy() must allow writes to devices on the lower file system.
- union_bwrite() was completely missing.

For devices, sockets and fifos ignore setting the file size to zero to make
open(..., O_TRUNC) happy and allow them to write through the lower layer.

Fixes PR #43560 (writing to null device in unionfs fails)

Change union rmdir semantics to fail directory removal for
non-empty directories like all other file systems do.

Change test accordingly.

When union_lookup() creates a shadow directory and nameiop is not LOOKUP
it has to restart the lookup to get the componentname right.

Fixes PR #44383 (an endless stream of whiteout and opaque dir problems ...)

Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

branches: 1.39.6;
Remove the special refcount behavior (adding an extra reference to the
parent dir) associated with SAVESTART in relookup().

Check all call sites to make sure that SAVESTART wasn't set while
calling relookup(); if it was, adjust the refcount behavior. Remove
related references to SAVESTART.

The only code that was reaching the extra ref was msdosfs_rename,
where the refcount behavior was already fairly broken and/or gross;
repair it.

Add a dummy 4th argument to relookup to make sure code that hasn't
been inspected won't compile. (This will go away next time the
relookup semantics change, which they will.)

LK_INTERLOCK is no longer a valid flag for VOP_LOCK().

Remove vlockmgr(). Generic vnode lock operations now use a rwlock located
in the vnode. All LK_* flags move from sys/lock.h to sys/vnode.h. Calls
to vlockmgr() in file systems get replaced with VOP_LOCK() or VOP_UNLOCK().

Welcome to 5.99.34.

Discussed on tech-kern.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

Return correct value from union_islocked.
(XXX: anything asserting LK_SHARED will fail because union doesn't
support shared locks)

branches: 1.34.2; 1.34.4;
The VATTR_NULL/VREF/VHOLD/HOLDRELE() macros lost their will to live
years ago when the kernel was modified to not alter ABI based on
DIAGNOSTIC, and now just call the respective function interfaces
(in lowercase). Plenty of mix'n match upper/lowercase has creeped
into the tree since then. Nuke the macros and convert all callsites
to lowercase.

no functional change

PR kern/38141 lookup/vfs_busy acquire rwlock recursively

Simplify the mount locking. Remove all the crud to deal with recursion on
the mount lock, and crud to deal with unmount as another weirdo lock.

Hopefully this will once and for all fix the deadlocks with this. With this
commit there are two locks on each mount:

- krwlock_t mnt_unmounting. This is used to prevent unmount across critical
sections like getnewvnode(). It's only ever read locked with rw_tryenter(),
and is only ever write locked in dounmount(). A write hold can't be taken
on this lock if the current LWP could hold a vnode lock.

- kmutex_t mnt_updating. This is taken by threads updating the mount, for
example when going r/o -> r/w, and is only present to serialize updates.
In order to take this lock, a read hold must first be taken on
mnt_unmounting, and the two need to be held across the operation.

One effect of this change: previously if an unmount failed, we would make a
half hearted attempt to back out of it gracefully, but that was unlikely to
work in a lot of cases. Now while an unmount that will be aborted is in
progress, new file operations within the mount will fail instead of being
delayed. That is unlikely to be a problem though, because if the admin
requests unmount of a file system then s(he) has made a decision to deny
access to the resource.

PR kern/38135 vfs_busy/vfs_trybusy confusion

The previous fix worked, but it opened a window where mounts could have
disappeared from mountlist while the caller was traversing it using
vfs_trybusy(). Fix that.

kern/38135 vfs_busy/vfs_trybusy confusion

The symptom was that sometimes file systems would occasionally not appear
in output from 'df' or 'mount' if the system was busy. Resolution:

- Make mount locks work somewhat like vm_map locks.
- vfs_trybusy() now only fails if the mount is gone, or if someone is
unmounting the file system. Simple contention on mnt_lock doesn't
cause it to fail.
- vfs_busy() will wait even if the file system is being unmounted.

branches: 1.30.2; 1.30.4;
Convert to ansi definitions from old-style definitons.

branches: 1.29.2; 1.29.6;
Restore a fragment of a comment that apparently got lost in a merge slipup
in 1998.

PR kern/37706 (forced unmount of file systems is unsafe):

- Do reference counting for 'struct mount'. Each vnode associated with a
mount takes a reference, and in turn the mount takes a reference to the
vfsops.
- Now that mounts are reference counted, replace the overcomplicated mount
locking inherited from 4.4BSD with a recursable rwlock.

Remove VOP_LEASE. Discussed on tech-kern.

union_putpages: fix a locking botch.

Merge vmlocking2 to head.

branches: 1.24.4;
Remove cn_lwp from struct componentname. curlwp should be used
from on. The NDINIT() macro no longer takes the lwp parameter and
associates the credentials of the calling thread with the namei
structure.

branches: 1.23.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.22.4;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

branches: 1.21.4; 1.21.6; 1.21.8; 1.21.10;
apply ketchup to vop_mmap_args comment

branches: 1.20.2;
define a pager flag PGO_RECLAIM, similar to FSYNC_RECLAIM, and use it
to skip unnecessary flushing when layered file system vnodes are recycled.
this also prevents a deadlock with the dodgy LFS putpages routine.
fixes the non-LFS part of PR 36150.

branches: 1.19.2; 1.19.6; 1.19.8;
more fixes for the new vnode locking scheme:
- don't use SAVESTART in calls to relookup() from unionfs,
just vref() the desired vnode when we need to.
- fix locking and refcounting in the unionfs EEXIST error cases.
- release any vnode locks before calling VFS_ROOT(), vfs_busy() is enough.
this allows us to simplify union_root() and fix PR 3006.
- union_lock() doesn't handle shared lock requests correctly,
so convert them to exclusive instead. fixes PR 34775.
- in relookup(), avoid reusing "dp" for different purposes,
the error handling wasn't right. (actually just get rid of dp.)
also, change relookup() to ignore LOCKLEAF and always return the
vnode locked since the callers already expect this.

a smorgasbord of improvements to vnode locking and path lookup:
- LOCKPARENT is no longer relevant for lookup(), relookup() or VOP_LOOKUP().
these now always return the parent vnode locked. namei() works as before.
lookup() and various other paths no longer acquire vnode locks in the
wrong order via vrele(). fixes PR 32535.
as a nice side effect, path lookup is also up to 25% faster.
- the above allows us to get rid of PDIRUNLOCK.
- also get rid of WANTPARENT (just use LOCKPARENT and unlock it).
- remove an assumption in layer_node_find() that all file systems implement
a recursive VOP_LOCK() (unionfs doesn't).
- require that all file systems supply vfs_vptofh and vfs_fhtovp routines.
fill in eopnotsupp() for file systems that don't support being exported
and remove the checks for NULL. (layerfs calls these without checking.)
- in union_lookup1(), don't change refcounts in the ISDOTDOT case, just
adjust which vnode is locked. fixes PR 33374.
- apply fixes for ufs_rename() from ufs_vnops.c rev. 1.61 to ext2fs_rename().

branches: 1.17.2;
Coverity CID 2960: Don't print NULL vnode (from Arnaud Lacombe)

branches: 1.16.8; 1.16.10;
integrate kauth.

Coverity CID 1001: Avoid NULL deref.

branches: 1.14.4; 1.14.6; 1.14.8; 1.14.10; 1.14.12;
merge ktrace-lwp.

branches: 1.13.2;
merge yamt-vop branch. remove following VOPs.

VOP_BLKATOFF
VOP_VALLOC
VOP_BALLOC
VOP_REALLOCBLKS
VOP_VFREE
VOP_TRUNCATE
VOP_UPDATE

branches: 1.12.2;
Remove __P()

branches: 1.11.4;
nuke trailing whitespace

branches: 1.10.4; 1.10.6;
First pass for some caddr_t removal and changes to get rid of it where we
no longer use and/or need it

- removed casts from unionfs, deadfs and fdesc
(there are more to hunt down still)
- changed vfs_quotactl args argumet from caddr_t to void *
- changed vfs_quotactl structures/callers to reflect the api change

Compiled fine and ran for about a day. Approved/reviewed by
christos@netbsd.org and gimpy@netbsd.org.

Replace the statfs() family of system calls with statvfs().
Retain binary compatibility.

Make VOP_STRATEGY(bp) a real VOP as discussed on tech-kern.

VOP_STRATEGY(bp) is replaced by one of two new functions:

- VOP_STRATEGY(vp, bp) Call the strategy routine of vp for bp.
- DEV_STRATEGY(bp) Call the d_strategy routine of bp->b_dev for bp.

DEV_STRATEGY(bp) is used only for block-to-block device situations.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.6.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Undo part of the ktrace/lwp changes. In particular:
* Remove the "lwp *" argument that was added to vget(). Turns out
that nothing actually used it!
* Remove the "lwp *" arguments that were added to VFS_ROOT(), VFS_VGET(),
and VFS_FHTOVP(); all they did was pass it to vget() (which, as noted
above, didn't use it).
* Remove all of the "lwp *" arguments to internal functions that were added
just to appease the above.

Fix problems with Darren's ktrace/lwp changes.

More changes for providing lwpid for ktrace (sparc GENERIC built)

add kqueue support

move union filesystem code from sys/miscfs/union to sys/fs/union

s/unaccessible/inaccessible/

Fix previous, don't copy up if the underlying node is unreadable.

In union_access() copy up regular files before checking permissions.

Unionfs is meant to provide a writable layer above a read-only layer
and should not fail here just because the lower layer is mounted
read-only.

Overhaul of the EVFILT_VNODE kevent(2) filter:

- Centralize vnode kevent handling in the VOP_*() wrappers, rather than
forcing each individual file system to deal with it (except VOP_RENAME(),
because VOP_RENAME() is a mess and we currently have 2 different ways
of handling it; at least it's reasonably well-centralized in the "new"
way).
- Add support for NOTE_OPEN, NOTE_CLOSE, NOTE_CLOSE_WRITE, and NOTE_READ,
compatible with the same events in FreeBSD.
- Track which kevent notifications clients are interested in receiving
to avoid doing work for events no one cares about (avoiding, e.g.
taking locks and traversing the klist to send a NOTE_WRITE when
someone is merely watching for a file to be deleted, for example).

In support of the above:

- Add support in vnode_if.sh for specifying PRE- and POST-op handlers,
to be invoked before and after vop_pre() and vop_post(), respectively.
Basic idea from FreeBSD, but implemented differently.
- Add support in vnode_if.sh for specifying CONTEXT fields in the
vop_*_args structures. These context fields are used to convey information
between the file system VOP function and the VOP wrapper, but do not
occupy an argument slot in the VOP_*() call itself. These context fields
are initialized and subsequently interpreted by PRE- and POST-op handlers.
- Version VOP_REMOVE(), uses the a context field for the file system to report
back the resulting link count of the target vnode. Return this in tmpfs,
udf, nfs, chfs, ext2fs, lfs, and ufs.

NetBSD 9.99.92.

Fix union_parsepath(), either the upper or the lower dvp may be NULL.

Now remove cn_consume from struct componentname.

This change requires a kernel bump.

Note though that I'm not going to version the VOP_LOOKUP args
structure (or any other args structure) as code that doesn't touch
cn_consume doesn't need attention and code that does will fail on it
without further intervention.

Onionfs needs to know about parsepath too, in case it has one of the
other cases underneath it.

The solution here is not really very good (take the longer
path-to-consume if they're different) but it will serve for the cases
that exist.

(If we were to add a fs that really uses different naming semantics,
we'd have to take additional steps; probably it doesn't make sense to
allow unionfs to union such a thing with a normal fs and attempting it
should fail at mount time.)

Update fs/unionfs as well to avoid increasing the current set of
compile failures there. Though maybe it's time to just remove
fs/unionfs.

- Add a new vnode op: VOP_PARSEPATH.
- Move namei_getcomponent to genfs_vnops.c and call it genfs_parsepath.
- Add a parsepath entry to every vnode ops table.

VOP_PARSEPATH takes a directory vnode to be searched and a complete
following path and chooses how much of that path to consume. To begin
with, all parsepath calls are genfs_parsepath, which locates the first
'/' as always.

Note that the call doesn't take the whole struct componentname, only
the string. The other bits of struct componentname should not be
needed and there's no reason to cause potential complications by
exposing them.

branches: 1.74.6;
Operation union_readdirhook() stores the lower directory as un_uppervp.
This breaks the assumption that un_uppervp->v_mount is the upper mount.

Fix by storing the directory as un_lowervp and adapt union_readdir().

Should fix PR kern/55552: panic with union mount

Add ACL support for FFS. From FreeBSD.

UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.70.10; 1.70.14; 1.70.16;
Make VOP_RECLAIM do the last unlock of the vnode.

VOP_RECLAIM naturally has exclusive access to the vnode, so having it
locked on entry is not strictly necessary -- but it means if there
are any final operations that must be done on the vnode, such as
ffs_update, requiring exclusive access to it, we can now kassert that
the vnode is locked in those operations.

We can't just have the caller release the last lock because some file
systems don't use genfs_lock, and require the vnode to remain valid
for VOP_UNLOCK to work, notably unionfs.

Use VCALL() to lock or unlock the lower node.

Move v_writecount adjustment from revoke to reclaim.

branches: 1.67.2;
Change VOP_REMOVE and VOP_RMDIR to preserve lock/ref on dvp.

No change to vp -- the plan is to replace the node by the
componentname in the vop parameters, and let all directory vops do
lookups internally.

Proposed on tech-kern with no objections:
https://mail-index.netbsd.org/tech-kern/2017/04/17/msg021825.html

Remove unused argument "nextp" from vfs_busy() and vfs_unbusy().
Remove argument "keepref" from vfs_unbusy() and add vfs_ref() where needed.

Make VOP_INACTIVE preserve vnode lock on return.

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2017/04/01/msg021751.html

Ride 7.99.68, a bumpy bus of incremental vfs improvements!

Handle v_writecount from union_open(), union_close() and union_revoke()
so lower file system vnodes get marked as open for writing.

branches: 1.63.2; 1.63.4;
Make VOP_LINK return directory still locked and referenced.

Ride 7.99.10 bump.

branches: 1.62.4;
Add VOP_FALLOCATE and VOP_FDISCARD to every vnode ops table I can
find.

The filesystem ones all call genfs_eopnotsupp - right now I am only
implementing the plumbing and we can implement fallocate and/or
fdiscard for files later.

The device ones call spec_fallocate (which is also genfs_eopnotsupp)
and spec_fdiscard, which dispatches to the device-level op.

The fifo ones all call vn_fifo_bypass, which also ends up being
EOPNOTSUPP.

Also set or assert that *vpp is null before calling VOP_MKDIR.

Set *vpp to NULL before calling VOP_CREATE. This always happens when
calling using nameidata, and if not something went wrong, so we'd like
to be able to assert about it.

branches: 1.59.2;
- Make VI_XLOCK, VI_CLEAN and VI_LOCKSHARE private to kern/vfs_*.c.
- Make vwait() static.
- Add vdead_check() to check a vnode for being or becoming dead.

Discussed on tech-kern.

Welcome to 6.99.38

Restructure union_lock() to always lock before testing for dead node.
Add two little helpers to lock or unlock a node. Use "vp" for the
union node and "lockvp" for the node to be locked. Use ISSET() to
test flags, add assertions.

The current implementation of vn_lock() is racy. Modification of
the vnode operations vector for active vnodes is unsafe because it
is not known whether deadfs or the original file system will be
called.

- Pass down LK_RETRY to the lock operation (hint for deadfs only).

- Change deadfs lock operation to return ENOENT if LK_RETRY is unset.

- Change all other lock operations to check for dead vnode once
the vnode is locked and unlock and return ENOENT in this case.

With these changes in place vnode lock operations will never succeed
after vclean() has marked the vnode as VI_XLOCK and before vclean()
has changed the operations vector.

Adresses PR kern/37706 (Forced unmount of file systems is unsafe)

Discussed on tech-kern.

Welcome to 6.99.33

Change union_allocvp() to take an unlocked uppervp and to return the
union node unlocked. Another VI_XLOCK hack is gone.

Remove an unused variable

Get rid of UN_KLOCK to keep a lock on vput(). It is not really needed
and makes the source difficult to read. Always hold references to the
union nodes until the operation is done.

Fix the DOT and DOTDOT case for union_lookup1().

Change vnode operation lookup to return the resulting vnode *vpp unlocked.
Change cache_lookup() to return an unlocked vnode.

Discussed on tech-kern@

Welcome to 6.99.31

Change vnode operations create, mknod, mkdir and symlink to return
the resulting vnode *vpp unlocked.

Discussed on tech-kern@

Welcome to 6.99.30

Change vnode operations create, mknod, mkdir and symlink to keep the
directory node dvp locked on return.

Discussed on tech-kern@

Welcome to 6.99.29

branches: 1.49.8; 1.49.12;
Replace flag based union node locking with generic vnode lock, support
shared and nowait locks and protect un_uppervp and un_*sz with mutex.

Mark file system MPSAFE.

VOP_ABORTOP() has no specific lock requirements so there is no need
to force locked vnodes here. It should be impossible to come here
with a nil upper node.

Relock the directory vnode after copyup. A locked union node with an
unlocked upper vnode can no longer exist so make FIXUP() an assertion.

branches: 1.47.2;
VOP_GETATTR() needs a shared lock at least.

Stop abusing relookup() to prepare the creation of new nodes
in the upper layer.
Replace union_relookup() with union_do_lookup() that prepares
a component, calls VOP_LOOKUP() and does the EEXIST test.

Change some `#ifdef DIAGNOSTIC' to `KASSERT'.
Instead of a `pid_t' use a `lwp_t *' for locking diagnostics.

No functional changes intended.

Add missing parts to mount devices from a union file system:
- union_close() has to lock/unlock the lower vnode.
- union_fsync() has to call spec_fsync() for the union vnode.
- union_strategy() must allow writes to devices on the lower file system.
- union_bwrite() was completely missing.

For devices, sockets and fifos ignore setting the file size to zero to make
open(..., O_TRUNC) happy and allow them to write through the lower layer.

Fixes PR #43560 (writing to null device in unionfs fails)

Change union rmdir semantics to fail directory removal for
non-empty directories like all other file systems do.

Change test accordingly.

When union_lookup() creates a shadow directory and nameiop is not LOOKUP
it has to restart the lookup to get the componentname right.

Fixes PR #44383 (an endless stream of whiteout and opaque dir problems ...)

Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

branches: 1.39.6;
Remove the special refcount behavior (adding an extra reference to the
parent dir) associated with SAVESTART in relookup().

Check all call sites to make sure that SAVESTART wasn't set while
calling relookup(); if it was, adjust the refcount behavior. Remove
related references to SAVESTART.

The only code that was reaching the extra ref was msdosfs_rename,
where the refcount behavior was already fairly broken and/or gross;
repair it.

Add a dummy 4th argument to relookup to make sure code that hasn't
been inspected won't compile. (This will go away next time the
relookup semantics change, which they will.)

LK_INTERLOCK is no longer a valid flag for VOP_LOCK().

Remove vlockmgr(). Generic vnode lock operations now use a rwlock located
in the vnode. All LK_* flags move from sys/lock.h to sys/vnode.h. Calls
to vlockmgr() in file systems get replaced with VOP_LOCK() or VOP_UNLOCK().

Welcome to 5.99.34.

Discussed on tech-kern.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

Return correct value from union_islocked.
(XXX: anything asserting LK_SHARED will fail because union doesn't
support shared locks)

branches: 1.34.2; 1.34.4;
The VATTR_NULL/VREF/VHOLD/HOLDRELE() macros lost their will to live
years ago when the kernel was modified to not alter ABI based on
DIAGNOSTIC, and now just call the respective function interfaces
(in lowercase). Plenty of mix'n match upper/lowercase has creeped
into the tree since then. Nuke the macros and convert all callsites
to lowercase.

no functional change

PR kern/38141 lookup/vfs_busy acquire rwlock recursively

Simplify the mount locking. Remove all the crud to deal with recursion on
the mount lock, and crud to deal with unmount as another weirdo lock.

Hopefully this will once and for all fix the deadlocks with this. With this
commit there are two locks on each mount:

- krwlock_t mnt_unmounting. This is used to prevent unmount across critical
sections like getnewvnode(). It's only ever read locked with rw_tryenter(),
and is only ever write locked in dounmount(). A write hold can't be taken
on this lock if the current LWP could hold a vnode lock.

- kmutex_t mnt_updating. This is taken by threads updating the mount, for
example when going r/o -> r/w, and is only present to serialize updates.
In order to take this lock, a read hold must first be taken on
mnt_unmounting, and the two need to be held across the operation.

One effect of this change: previously if an unmount failed, we would make a
half hearted attempt to back out of it gracefully, but that was unlikely to
work in a lot of cases. Now while an unmount that will be aborted is in
progress, new file operations within the mount will fail instead of being
delayed. That is unlikely to be a problem though, because if the admin
requests unmount of a file system then s(he) has made a decision to deny
access to the resource.

PR kern/38135 vfs_busy/vfs_trybusy confusion

The previous fix worked, but it opened a window where mounts could have
disappeared from mountlist while the caller was traversing it using
vfs_trybusy(). Fix that.

kern/38135 vfs_busy/vfs_trybusy confusion

The symptom was that sometimes file systems would occasionally not appear
in output from 'df' or 'mount' if the system was busy. Resolution:

- Make mount locks work somewhat like vm_map locks.
- vfs_trybusy() now only fails if the mount is gone, or if someone is
unmounting the file system. Simple contention on mnt_lock doesn't
cause it to fail.
- vfs_busy() will wait even if the file system is being unmounted.

branches: 1.30.2; 1.30.4;
Convert to ansi definitions from old-style definitons.

branches: 1.29.2; 1.29.6;
Restore a fragment of a comment that apparently got lost in a merge slipup
in 1998.

PR kern/37706 (forced unmount of file systems is unsafe):

- Do reference counting for 'struct mount'. Each vnode associated with a
mount takes a reference, and in turn the mount takes a reference to the
vfsops.
- Now that mounts are reference counted, replace the overcomplicated mount
locking inherited from 4.4BSD with a recursable rwlock.

Remove VOP_LEASE. Discussed on tech-kern.

union_putpages: fix a locking botch.

Merge vmlocking2 to head.

branches: 1.24.4;
Remove cn_lwp from struct componentname. curlwp should be used
from on. The NDINIT() macro no longer takes the lwp parameter and
associates the credentials of the calling thread with the namei
structure.

branches: 1.23.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.22.4;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

branches: 1.21.4; 1.21.6; 1.21.8; 1.21.10;
apply ketchup to vop_mmap_args comment

branches: 1.20.2;
define a pager flag PGO_RECLAIM, similar to FSYNC_RECLAIM, and use it
to skip unnecessary flushing when layered file system vnodes are recycled.
this also prevents a deadlock with the dodgy LFS putpages routine.
fixes the non-LFS part of PR 36150.

branches: 1.19.2; 1.19.6; 1.19.8;
more fixes for the new vnode locking scheme:
- don't use SAVESTART in calls to relookup() from unionfs,
just vref() the desired vnode when we need to.
- fix locking and refcounting in the unionfs EEXIST error cases.
- release any vnode locks before calling VFS_ROOT(), vfs_busy() is enough.
this allows us to simplify union_root() and fix PR 3006.
- union_lock() doesn't handle shared lock requests correctly,
so convert them to exclusive instead. fixes PR 34775.
- in relookup(), avoid reusing "dp" for different purposes,
the error handling wasn't right. (actually just get rid of dp.)
also, change relookup() to ignore LOCKLEAF and always return the
vnode locked since the callers already expect this.

a smorgasbord of improvements to vnode locking and path lookup:
- LOCKPARENT is no longer relevant for lookup(), relookup() or VOP_LOOKUP().
these now always return the parent vnode locked. namei() works as before.
lookup() and various other paths no longer acquire vnode locks in the
wrong order via vrele(). fixes PR 32535.
as a nice side effect, path lookup is also up to 25% faster.
- the above allows us to get rid of PDIRUNLOCK.
- also get rid of WANTPARENT (just use LOCKPARENT and unlock it).
- remove an assumption in layer_node_find() that all file systems implement
a recursive VOP_LOCK() (unionfs doesn't).
- require that all file systems supply vfs_vptofh and vfs_fhtovp routines.
fill in eopnotsupp() for file systems that don't support being exported
and remove the checks for NULL. (layerfs calls these without checking.)
- in union_lookup1(), don't change refcounts in the ISDOTDOT case, just
adjust which vnode is locked. fixes PR 33374.
- apply fixes for ufs_rename() from ufs_vnops.c rev. 1.61 to ext2fs_rename().

branches: 1.17.2;
Coverity CID 2960: Don't print NULL vnode (from Arnaud Lacombe)

branches: 1.16.8; 1.16.10;
integrate kauth.

Coverity CID 1001: Avoid NULL deref.

branches: 1.14.4; 1.14.6; 1.14.8; 1.14.10; 1.14.12;
merge ktrace-lwp.

branches: 1.13.2;
merge yamt-vop branch. remove following VOPs.

VOP_BLKATOFF
VOP_VALLOC
VOP_BALLOC
VOP_REALLOCBLKS
VOP_VFREE
VOP_TRUNCATE
VOP_UPDATE

branches: 1.12.2;
Remove __P()

branches: 1.11.4;
nuke trailing whitespace

branches: 1.10.4; 1.10.6;
First pass for some caddr_t removal and changes to get rid of it where we
no longer use and/or need it

- removed casts from unionfs, deadfs and fdesc
(there are more to hunt down still)
- changed vfs_quotactl args argumet from caddr_t to void *
- changed vfs_quotactl structures/callers to reflect the api change

Compiled fine and ran for about a day. Approved/reviewed by
christos@netbsd.org and gimpy@netbsd.org.

Replace the statfs() family of system calls with statvfs().
Retain binary compatibility.

Make VOP_STRATEGY(bp) a real VOP as discussed on tech-kern.

VOP_STRATEGY(bp) is replaced by one of two new functions:

- VOP_STRATEGY(vp, bp) Call the strategy routine of vp for bp.
- DEV_STRATEGY(bp) Call the d_strategy routine of bp->b_dev for bp.

DEV_STRATEGY(bp) is used only for block-to-block device situations.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.6.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Undo part of the ktrace/lwp changes. In particular:
* Remove the "lwp *" argument that was added to vget(). Turns out
that nothing actually used it!
* Remove the "lwp *" arguments that were added to VFS_ROOT(), VFS_VGET(),
and VFS_FHTOVP(); all they did was pass it to vget() (which, as noted
above, didn't use it).
* Remove all of the "lwp *" arguments to internal functions that were added
just to appease the above.

Fix problems with Darren's ktrace/lwp changes.

More changes for providing lwpid for ktrace (sparc GENERIC built)

add kqueue support

move union filesystem code from sys/miscfs/union to sys/fs/union

Fix previous, don't copy up if the underlying node is unreadable.

In union_access() copy up regular files before checking permissions.

Unionfs is meant to provide a writable layer above a read-only layer
and should not fail here just because the lower layer is mounted
read-only.

Overhaul of the EVFILT_VNODE kevent(2) filter:

- Centralize vnode kevent handling in the VOP_*() wrappers, rather than
forcing each individual file system to deal with it (except VOP_RENAME(),
because VOP_RENAME() is a mess and we currently have 2 different ways
of handling it; at least it's reasonably well-centralized in the "new"
way).
- Add support for NOTE_OPEN, NOTE_CLOSE, NOTE_CLOSE_WRITE, and NOTE_READ,
compatible with the same events in FreeBSD.
- Track which kevent notifications clients are interested in receiving
to avoid doing work for events no one cares about (avoiding, e.g.
taking locks and traversing the klist to send a NOTE_WRITE when
someone is merely watching for a file to be deleted, for example).

In support of the above:

- Add support in vnode_if.sh for specifying PRE- and POST-op handlers,
to be invoked before and after vop_pre() and vop_post(), respectively.
Basic idea from FreeBSD, but implemented differently.
- Add support in vnode_if.sh for specifying CONTEXT fields in the
vop_*_args structures. These context fields are used to convey information
between the file system VOP function and the VOP wrapper, but do not
occupy an argument slot in the VOP_*() call itself. These context fields
are initialized and subsequently interpreted by PRE- and POST-op handlers.
- Version VOP_REMOVE(), uses the a context field for the file system to report
back the resulting link count of the target vnode. Return this in tmpfs,
udf, nfs, chfs, ext2fs, lfs, and ufs.

NetBSD 9.99.92.

Fix union_parsepath(), either the upper or the lower dvp may be NULL.

Now remove cn_consume from struct componentname.

This change requires a kernel bump.

Note though that I'm not going to version the VOP_LOOKUP args
structure (or any other args structure) as code that doesn't touch
cn_consume doesn't need attention and code that does will fail on it
without further intervention.

Onionfs needs to know about parsepath too, in case it has one of the
other cases underneath it.

The solution here is not really very good (take the longer
path-to-consume if they're different) but it will serve for the cases
that exist.

(If we were to add a fs that really uses different naming semantics,
we'd have to take additional steps; probably it doesn't make sense to
allow unionfs to union such a thing with a normal fs and attempting it
should fail at mount time.)

Update fs/unionfs as well to avoid increasing the current set of
compile failures there. Though maybe it's time to just remove
fs/unionfs.

- Add a new vnode op: VOP_PARSEPATH.
- Move namei_getcomponent to genfs_vnops.c and call it genfs_parsepath.
- Add a parsepath entry to every vnode ops table.

VOP_PARSEPATH takes a directory vnode to be searched and a complete
following path and chooses how much of that path to consume. To begin
with, all parsepath calls are genfs_parsepath, which locates the first
'/' as always.

Note that the call doesn't take the whole struct componentname, only
the string. The other bits of struct componentname should not be
needed and there's no reason to cause potential complications by
exposing them.

branches: 1.74.6;
Operation union_readdirhook() stores the lower directory as un_uppervp.
This breaks the assumption that un_uppervp->v_mount is the upper mount.

Fix by storing the directory as un_lowervp and adapt union_readdir().

Should fix PR kern/55552: panic with union mount

Add ACL support for FFS. From FreeBSD.

UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.70.10; 1.70.14; 1.70.16;
Make VOP_RECLAIM do the last unlock of the vnode.

VOP_RECLAIM naturally has exclusive access to the vnode, so having it
locked on entry is not strictly necessary -- but it means if there
are any final operations that must be done on the vnode, such as
ffs_update, requiring exclusive access to it, we can now kassert that
the vnode is locked in those operations.

We can't just have the caller release the last lock because some file
systems don't use genfs_lock, and require the vnode to remain valid
for VOP_UNLOCK to work, notably unionfs.

Use VCALL() to lock or unlock the lower node.

Move v_writecount adjustment from revoke to reclaim.

branches: 1.67.2;
Change VOP_REMOVE and VOP_RMDIR to preserve lock/ref on dvp.

No change to vp -- the plan is to replace the node by the
componentname in the vop parameters, and let all directory vops do
lookups internally.

Proposed on tech-kern with no objections:
https://mail-index.netbsd.org/tech-kern/2017/04/17/msg021825.html

Remove unused argument "nextp" from vfs_busy() and vfs_unbusy().
Remove argument "keepref" from vfs_unbusy() and add vfs_ref() where needed.

Make VOP_INACTIVE preserve vnode lock on return.

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2017/04/01/msg021751.html

Ride 7.99.68, a bumpy bus of incremental vfs improvements!

Handle v_writecount from union_open(), union_close() and union_revoke()
so lower file system vnodes get marked as open for writing.

branches: 1.63.2; 1.63.4;
Make VOP_LINK return directory still locked and referenced.

Ride 7.99.10 bump.

branches: 1.62.4;
Add VOP_FALLOCATE and VOP_FDISCARD to every vnode ops table I can
find.

The filesystem ones all call genfs_eopnotsupp - right now I am only
implementing the plumbing and we can implement fallocate and/or
fdiscard for files later.

The device ones call spec_fallocate (which is also genfs_eopnotsupp)
and spec_fdiscard, which dispatches to the device-level op.

The fifo ones all call vn_fifo_bypass, which also ends up being
EOPNOTSUPP.

Also set or assert that *vpp is null before calling VOP_MKDIR.

Set *vpp to NULL before calling VOP_CREATE. This always happens when
calling using nameidata, and if not something went wrong, so we'd like
to be able to assert about it.

branches: 1.59.2;
- Make VI_XLOCK, VI_CLEAN and VI_LOCKSHARE private to kern/vfs_*.c.
- Make vwait() static.
- Add vdead_check() to check a vnode for being or becoming dead.

Discussed on tech-kern.

Welcome to 6.99.38

Restructure union_lock() to always lock before testing for dead node.
Add two little helpers to lock or unlock a node. Use "vp" for the
union node and "lockvp" for the node to be locked. Use ISSET() to
test flags, add assertions.

The current implementation of vn_lock() is racy. Modification of
the vnode operations vector for active vnodes is unsafe because it
is not known whether deadfs or the original file system will be
called.

- Pass down LK_RETRY to the lock operation (hint for deadfs only).

- Change deadfs lock operation to return ENOENT if LK_RETRY is unset.

- Change all other lock operations to check for dead vnode once
the vnode is locked and unlock and return ENOENT in this case.

With these changes in place vnode lock operations will never succeed
after vclean() has marked the vnode as VI_XLOCK and before vclean()
has changed the operations vector.

Adresses PR kern/37706 (Forced unmount of file systems is unsafe)

Discussed on tech-kern.

Welcome to 6.99.33

Change union_allocvp() to take an unlocked uppervp and to return the
union node unlocked. Another VI_XLOCK hack is gone.

Remove an unused variable

Get rid of UN_KLOCK to keep a lock on vput(). It is not really needed
and makes the source difficult to read. Always hold references to the
union nodes until the operation is done.

Fix the DOT and DOTDOT case for union_lookup1().

Change vnode operation lookup to return the resulting vnode *vpp unlocked.
Change cache_lookup() to return an unlocked vnode.

Discussed on tech-kern@

Welcome to 6.99.31

Change vnode operations create, mknod, mkdir and symlink to return
the resulting vnode *vpp unlocked.

Discussed on tech-kern@

Welcome to 6.99.30

Change vnode operations create, mknod, mkdir and symlink to keep the
directory node dvp locked on return.

Discussed on tech-kern@

Welcome to 6.99.29

branches: 1.49.8; 1.49.12;
Replace flag based union node locking with generic vnode lock, support
shared and nowait locks and protect un_uppervp and un_*sz with mutex.

Mark file system MPSAFE.

VOP_ABORTOP() has no specific lock requirements so there is no need
to force locked vnodes here. It should be impossible to come here
with a nil upper node.

Relock the directory vnode after copyup. A locked union node with an
unlocked upper vnode can no longer exist so make FIXUP() an assertion.

branches: 1.47.2;
VOP_GETATTR() needs a shared lock at least.

Stop abusing relookup() to prepare the creation of new nodes
in the upper layer.
Replace union_relookup() with union_do_lookup() that prepares
a component, calls VOP_LOOKUP() and does the EEXIST test.

Change some `#ifdef DIAGNOSTIC' to `KASSERT'.
Instead of a `pid_t' use a `lwp_t *' for locking diagnostics.

No functional changes intended.

Add missing parts to mount devices from a union file system:
- union_close() has to lock/unlock the lower vnode.
- union_fsync() has to call spec_fsync() for the union vnode.
- union_strategy() must allow writes to devices on the lower file system.
- union_bwrite() was completely missing.

For devices, sockets and fifos ignore setting the file size to zero to make
open(..., O_TRUNC) happy and allow them to write through the lower layer.

Fixes PR #43560 (writing to null device in unionfs fails)

Change union rmdir semantics to fail directory removal for
non-empty directories like all other file systems do.

Change test accordingly.

When union_lookup() creates a shadow directory and nameiop is not LOOKUP
it has to restart the lookup to get the componentname right.

Fixes PR #44383 (an endless stream of whiteout and opaque dir problems ...)

Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

branches: 1.39.6;
Remove the special refcount behavior (adding an extra reference to the
parent dir) associated with SAVESTART in relookup().

Check all call sites to make sure that SAVESTART wasn't set while
calling relookup(); if it was, adjust the refcount behavior. Remove
related references to SAVESTART.

The only code that was reaching the extra ref was msdosfs_rename,
where the refcount behavior was already fairly broken and/or gross;
repair it.

Add a dummy 4th argument to relookup to make sure code that hasn't
been inspected won't compile. (This will go away next time the
relookup semantics change, which they will.)

LK_INTERLOCK is no longer a valid flag for VOP_LOCK().

Remove vlockmgr(). Generic vnode lock operations now use a rwlock located
in the vnode. All LK_* flags move from sys/lock.h to sys/vnode.h. Calls
to vlockmgr() in file systems get replaced with VOP_LOCK() or VOP_UNLOCK().

Welcome to 5.99.34.

Discussed on tech-kern.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

Return correct value from union_islocked.
(XXX: anything asserting LK_SHARED will fail because union doesn't
support shared locks)

branches: 1.34.2; 1.34.4;
The VATTR_NULL/VREF/VHOLD/HOLDRELE() macros lost their will to live
years ago when the kernel was modified to not alter ABI based on
DIAGNOSTIC, and now just call the respective function interfaces
(in lowercase). Plenty of mix'n match upper/lowercase has creeped
into the tree since then. Nuke the macros and convert all callsites
to lowercase.

no functional change

PR kern/38141 lookup/vfs_busy acquire rwlock recursively

Simplify the mount locking. Remove all the crud to deal with recursion on
the mount lock, and crud to deal with unmount as another weirdo lock.

Hopefully this will once and for all fix the deadlocks with this. With this
commit there are two locks on each mount:

- krwlock_t mnt_unmounting. This is used to prevent unmount across critical
sections like getnewvnode(). It's only ever read locked with rw_tryenter(),
and is only ever write locked in dounmount(). A write hold can't be taken
on this lock if the current LWP could hold a vnode lock.

- kmutex_t mnt_updating. This is taken by threads updating the mount, for
example when going r/o -> r/w, and is only present to serialize updates.
In order to take this lock, a read hold must first be taken on
mnt_unmounting, and the two need to be held across the operation.

One effect of this change: previously if an unmount failed, we would make a
half hearted attempt to back out of it gracefully, but that was unlikely to
work in a lot of cases. Now while an unmount that will be aborted is in
progress, new file operations within the mount will fail instead of being
delayed. That is unlikely to be a problem though, because if the admin
requests unmount of a file system then s(he) has made a decision to deny
access to the resource.

PR kern/38135 vfs_busy/vfs_trybusy confusion

The previous fix worked, but it opened a window where mounts could have
disappeared from mountlist while the caller was traversing it using
vfs_trybusy(). Fix that.

kern/38135 vfs_busy/vfs_trybusy confusion

The symptom was that sometimes file systems would occasionally not appear
in output from 'df' or 'mount' if the system was busy. Resolution:

- Make mount locks work somewhat like vm_map locks.
- vfs_trybusy() now only fails if the mount is gone, or if someone is
unmounting the file system. Simple contention on mnt_lock doesn't
cause it to fail.
- vfs_busy() will wait even if the file system is being unmounted.

branches: 1.30.2; 1.30.4;
Convert to ansi definitions from old-style definitons.

branches: 1.29.2; 1.29.6;
Restore a fragment of a comment that apparently got lost in a merge slipup
in 1998.

PR kern/37706 (forced unmount of file systems is unsafe):

- Do reference counting for 'struct mount'. Each vnode associated with a
mount takes a reference, and in turn the mount takes a reference to the
vfsops.
- Now that mounts are reference counted, replace the overcomplicated mount
locking inherited from 4.4BSD with a recursable rwlock.

Remove VOP_LEASE. Discussed on tech-kern.

union_putpages: fix a locking botch.

Merge vmlocking2 to head.

branches: 1.24.4;
Remove cn_lwp from struct componentname. curlwp should be used
from on. The NDINIT() macro no longer takes the lwp parameter and
associates the credentials of the calling thread with the namei
structure.

branches: 1.23.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.22.4;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

branches: 1.21.4; 1.21.6; 1.21.8; 1.21.10;
apply ketchup to vop_mmap_args comment

branches: 1.20.2;
define a pager flag PGO_RECLAIM, similar to FSYNC_RECLAIM, and use it
to skip unnecessary flushing when layered file system vnodes are recycled.
this also prevents a deadlock with the dodgy LFS putpages routine.
fixes the non-LFS part of PR 36150.

branches: 1.19.2; 1.19.6; 1.19.8;
more fixes for the new vnode locking scheme:
- don't use SAVESTART in calls to relookup() from unionfs,
just vref() the desired vnode when we need to.
- fix locking and refcounting in the unionfs EEXIST error cases.
- release any vnode locks before calling VFS_ROOT(), vfs_busy() is enough.
this allows us to simplify union_root() and fix PR 3006.
- union_lock() doesn't handle shared lock requests correctly,
so convert them to exclusive instead. fixes PR 34775.
- in relookup(), avoid reusing "dp" for different purposes,
the error handling wasn't right. (actually just get rid of dp.)
also, change relookup() to ignore LOCKLEAF and always return the
vnode locked since the callers already expect this.

a smorgasbord of improvements to vnode locking and path lookup:
- LOCKPARENT is no longer relevant for lookup(), relookup() or VOP_LOOKUP().
these now always return the parent vnode locked. namei() works as before.
lookup() and various other paths no longer acquire vnode locks in the
wrong order via vrele(). fixes PR 32535.
as a nice side effect, path lookup is also up to 25% faster.
- the above allows us to get rid of PDIRUNLOCK.
- also get rid of WANTPARENT (just use LOCKPARENT and unlock it).
- remove an assumption in layer_node_find() that all file systems implement
a recursive VOP_LOCK() (unionfs doesn't).
- require that all file systems supply vfs_vptofh and vfs_fhtovp routines.
fill in eopnotsupp() for file systems that don't support being exported
and remove the checks for NULL. (layerfs calls these without checking.)
- in union_lookup1(), don't change refcounts in the ISDOTDOT case, just
adjust which vnode is locked. fixes PR 33374.
- apply fixes for ufs_rename() from ufs_vnops.c rev. 1.61 to ext2fs_rename().

branches: 1.17.2;
Coverity CID 2960: Don't print NULL vnode (from Arnaud Lacombe)

branches: 1.16.8; 1.16.10;
integrate kauth.

Coverity CID 1001: Avoid NULL deref.

branches: 1.14.4; 1.14.6; 1.14.8; 1.14.10; 1.14.12;
merge ktrace-lwp.

branches: 1.13.2;
merge yamt-vop branch. remove following VOPs.

VOP_BLKATOFF
VOP_VALLOC
VOP_BALLOC
VOP_REALLOCBLKS
VOP_VFREE
VOP_TRUNCATE
VOP_UPDATE

branches: 1.12.2;
Remove __P()

branches: 1.11.4;
nuke trailing whitespace

branches: 1.10.4; 1.10.6;
First pass for some caddr_t removal and changes to get rid of it where we
no longer use and/or need it

- removed casts from unionfs, deadfs and fdesc
(there are more to hunt down still)
- changed vfs_quotactl args argumet from caddr_t to void *
- changed vfs_quotactl structures/callers to reflect the api change

Compiled fine and ran for about a day. Approved/reviewed by
christos@netbsd.org and gimpy@netbsd.org.

Replace the statfs() family of system calls with statvfs().
Retain binary compatibility.

Make VOP_STRATEGY(bp) a real VOP as discussed on tech-kern.

VOP_STRATEGY(bp) is replaced by one of two new functions:

- VOP_STRATEGY(vp, bp) Call the strategy routine of vp for bp.
- DEV_STRATEGY(bp) Call the d_strategy routine of bp->b_dev for bp.

DEV_STRATEGY(bp) is used only for block-to-block device situations.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.6.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Undo part of the ktrace/lwp changes. In particular:
* Remove the "lwp *" argument that was added to vget(). Turns out
that nothing actually used it!
* Remove the "lwp *" arguments that were added to VFS_ROOT(), VFS_VGET(),
and VFS_FHTOVP(); all they did was pass it to vget() (which, as noted
above, didn't use it).
* Remove all of the "lwp *" arguments to internal functions that were added
just to appease the above.

Fix problems with Darren's ktrace/lwp changes.

More changes for providing lwpid for ktrace (sparc GENERIC built)

add kqueue support

move union filesystem code from sys/miscfs/union to sys/fs/union

In union_access() copy up regular files before checking permissions.

Unionfs is meant to provide a writable layer above a read-only layer
and should not fail here just because the lower layer is mounted
read-only.

Overhaul of the EVFILT_VNODE kevent(2) filter:

- Centralize vnode kevent handling in the VOP_*() wrappers, rather than
forcing each individual file system to deal with it (except VOP_RENAME(),
because VOP_RENAME() is a mess and we currently have 2 different ways
of handling it; at least it's reasonably well-centralized in the "new"
way).
- Add support for NOTE_OPEN, NOTE_CLOSE, NOTE_CLOSE_WRITE, and NOTE_READ,
compatible with the same events in FreeBSD.
- Track which kevent notifications clients are interested in receiving
to avoid doing work for events no one cares about (avoiding, e.g.
taking locks and traversing the klist to send a NOTE_WRITE when
someone is merely watching for a file to be deleted, for example).

In support of the above:

- Add support in vnode_if.sh for specifying PRE- and POST-op handlers,
to be invoked before and after vop_pre() and vop_post(), respectively.
Basic idea from FreeBSD, but implemented differently.
- Add support in vnode_if.sh for specifying CONTEXT fields in the
vop_*_args structures. These context fields are used to convey information
between the file system VOP function and the VOP wrapper, but do not
occupy an argument slot in the VOP_*() call itself. These context fields
are initialized and subsequently interpreted by PRE- and POST-op handlers.
- Version VOP_REMOVE(), uses the a context field for the file system to report
back the resulting link count of the target vnode. Return this in tmpfs,
udf, nfs, chfs, ext2fs, lfs, and ufs.

NetBSD 9.99.92.

Fix union_parsepath(), either the upper or the lower dvp may be NULL.

Now remove cn_consume from struct componentname.

This change requires a kernel bump.

Note though that I'm not going to version the VOP_LOOKUP args
structure (or any other args structure) as code that doesn't touch
cn_consume doesn't need attention and code that does will fail on it
without further intervention.

Onionfs needs to know about parsepath too, in case it has one of the
other cases underneath it.

The solution here is not really very good (take the longer
path-to-consume if they're different) but it will serve for the cases
that exist.

(If we were to add a fs that really uses different naming semantics,
we'd have to take additional steps; probably it doesn't make sense to
allow unionfs to union such a thing with a normal fs and attempting it
should fail at mount time.)

Update fs/unionfs as well to avoid increasing the current set of
compile failures there. Though maybe it's time to just remove
fs/unionfs.

- Add a new vnode op: VOP_PARSEPATH.
- Move namei_getcomponent to genfs_vnops.c and call it genfs_parsepath.
- Add a parsepath entry to every vnode ops table.

VOP_PARSEPATH takes a directory vnode to be searched and a complete
following path and chooses how much of that path to consume. To begin
with, all parsepath calls are genfs_parsepath, which locates the first
'/' as always.

Note that the call doesn't take the whole struct componentname, only
the string. The other bits of struct componentname should not be
needed and there's no reason to cause potential complications by
exposing them.

branches: 1.74.6;
Operation union_readdirhook() stores the lower directory as un_uppervp.
This breaks the assumption that un_uppervp->v_mount is the upper mount.

Fix by storing the directory as un_lowervp and adapt union_readdir().

Should fix PR kern/55552: panic with union mount

Add ACL support for FFS. From FreeBSD.

UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.70.10; 1.70.14; 1.70.16;
Make VOP_RECLAIM do the last unlock of the vnode.

VOP_RECLAIM naturally has exclusive access to the vnode, so having it
locked on entry is not strictly necessary -- but it means if there
are any final operations that must be done on the vnode, such as
ffs_update, requiring exclusive access to it, we can now kassert that
the vnode is locked in those operations.

We can't just have the caller release the last lock because some file
systems don't use genfs_lock, and require the vnode to remain valid
for VOP_UNLOCK to work, notably unionfs.

Use VCALL() to lock or unlock the lower node.

Move v_writecount adjustment from revoke to reclaim.

branches: 1.67.2;
Change VOP_REMOVE and VOP_RMDIR to preserve lock/ref on dvp.

No change to vp -- the plan is to replace the node by the
componentname in the vop parameters, and let all directory vops do
lookups internally.

Proposed on tech-kern with no objections:
https://mail-index.netbsd.org/tech-kern/2017/04/17/msg021825.html

Remove unused argument "nextp" from vfs_busy() and vfs_unbusy().
Remove argument "keepref" from vfs_unbusy() and add vfs_ref() where needed.

Make VOP_INACTIVE preserve vnode lock on return.

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2017/04/01/msg021751.html

Ride 7.99.68, a bumpy bus of incremental vfs improvements!

Handle v_writecount from union_open(), union_close() and union_revoke()
so lower file system vnodes get marked as open for writing.

branches: 1.63.2; 1.63.4;
Make VOP_LINK return directory still locked and referenced.

Ride 7.99.10 bump.

branches: 1.62.4;
Add VOP_FALLOCATE and VOP_FDISCARD to every vnode ops table I can
find.

The filesystem ones all call genfs_eopnotsupp - right now I am only
implementing the plumbing and we can implement fallocate and/or
fdiscard for files later.

The device ones call spec_fallocate (which is also genfs_eopnotsupp)
and spec_fdiscard, which dispatches to the device-level op.

The fifo ones all call vn_fifo_bypass, which also ends up being
EOPNOTSUPP.

Also set or assert that *vpp is null before calling VOP_MKDIR.

Set *vpp to NULL before calling VOP_CREATE. This always happens when
calling using nameidata, and if not something went wrong, so we'd like
to be able to assert about it.

branches: 1.59.2;
- Make VI_XLOCK, VI_CLEAN and VI_LOCKSHARE private to kern/vfs_*.c.
- Make vwait() static.
- Add vdead_check() to check a vnode for being or becoming dead.

Discussed on tech-kern.

Welcome to 6.99.38

Restructure union_lock() to always lock before testing for dead node.
Add two little helpers to lock or unlock a node. Use "vp" for the
union node and "lockvp" for the node to be locked. Use ISSET() to
test flags, add assertions.

The current implementation of vn_lock() is racy. Modification of
the vnode operations vector for active vnodes is unsafe because it
is not known whether deadfs or the original file system will be
called.

- Pass down LK_RETRY to the lock operation (hint for deadfs only).

- Change deadfs lock operation to return ENOENT if LK_RETRY is unset.

- Change all other lock operations to check for dead vnode once
the vnode is locked and unlock and return ENOENT in this case.

With these changes in place vnode lock operations will never succeed
after vclean() has marked the vnode as VI_XLOCK and before vclean()
has changed the operations vector.

Adresses PR kern/37706 (Forced unmount of file systems is unsafe)

Discussed on tech-kern.

Welcome to 6.99.33

Change union_allocvp() to take an unlocked uppervp and to return the
union node unlocked. Another VI_XLOCK hack is gone.

Remove an unused variable

Get rid of UN_KLOCK to keep a lock on vput(). It is not really needed
and makes the source difficult to read. Always hold references to the
union nodes until the operation is done.

Fix the DOT and DOTDOT case for union_lookup1().

Change vnode operation lookup to return the resulting vnode *vpp unlocked.
Change cache_lookup() to return an unlocked vnode.

Discussed on tech-kern@

Welcome to 6.99.31

Change vnode operations create, mknod, mkdir and symlink to return
the resulting vnode *vpp unlocked.

Discussed on tech-kern@

Welcome to 6.99.30

Change vnode operations create, mknod, mkdir and symlink to keep the
directory node dvp locked on return.

Discussed on tech-kern@

Welcome to 6.99.29

branches: 1.49.8; 1.49.12;
Replace flag based union node locking with generic vnode lock, support
shared and nowait locks and protect un_uppervp and un_*sz with mutex.

Mark file system MPSAFE.

VOP_ABORTOP() has no specific lock requirements so there is no need
to force locked vnodes here. It should be impossible to come here
with a nil upper node.

Relock the directory vnode after copyup. A locked union node with an
unlocked upper vnode can no longer exist so make FIXUP() an assertion.

branches: 1.47.2;
VOP_GETATTR() needs a shared lock at least.

Stop abusing relookup() to prepare the creation of new nodes
in the upper layer.
Replace union_relookup() with union_do_lookup() that prepares
a component, calls VOP_LOOKUP() and does the EEXIST test.

Change some `#ifdef DIAGNOSTIC' to `KASSERT'.
Instead of a `pid_t' use a `lwp_t *' for locking diagnostics.

No functional changes intended.

Add missing parts to mount devices from a union file system:
- union_close() has to lock/unlock the lower vnode.
- union_fsync() has to call spec_fsync() for the union vnode.
- union_strategy() must allow writes to devices on the lower file system.
- union_bwrite() was completely missing.

For devices, sockets and fifos ignore setting the file size to zero to make
open(..., O_TRUNC) happy and allow them to write through the lower layer.

Fixes PR #43560 (writing to null device in unionfs fails)

Change union rmdir semantics to fail directory removal for
non-empty directories like all other file systems do.

Change test accordingly.

When union_lookup() creates a shadow directory and nameiop is not LOOKUP
it has to restart the lookup to get the componentname right.

Fixes PR #44383 (an endless stream of whiteout and opaque dir problems ...)

Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

branches: 1.39.6;
Remove the special refcount behavior (adding an extra reference to the
parent dir) associated with SAVESTART in relookup().

Check all call sites to make sure that SAVESTART wasn't set while
calling relookup(); if it was, adjust the refcount behavior. Remove
related references to SAVESTART.

The only code that was reaching the extra ref was msdosfs_rename,
where the refcount behavior was already fairly broken and/or gross;
repair it.

Add a dummy 4th argument to relookup to make sure code that hasn't
been inspected won't compile. (This will go away next time the
relookup semantics change, which they will.)

LK_INTERLOCK is no longer a valid flag for VOP_LOCK().

Remove vlockmgr(). Generic vnode lock operations now use a rwlock located
in the vnode. All LK_* flags move from sys/lock.h to sys/vnode.h. Calls
to vlockmgr() in file systems get replaced with VOP_LOCK() or VOP_UNLOCK().

Welcome to 5.99.34.

Discussed on tech-kern.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

Return correct value from union_islocked.
(XXX: anything asserting LK_SHARED will fail because union doesn't
support shared locks)

branches: 1.34.2; 1.34.4;
The VATTR_NULL/VREF/VHOLD/HOLDRELE() macros lost their will to live
years ago when the kernel was modified to not alter ABI based on
DIAGNOSTIC, and now just call the respective function interfaces
(in lowercase). Plenty of mix'n match upper/lowercase has creeped
into the tree since then. Nuke the macros and convert all callsites
to lowercase.

no functional change

PR kern/38141 lookup/vfs_busy acquire rwlock recursively

Simplify the mount locking. Remove all the crud to deal with recursion on
the mount lock, and crud to deal with unmount as another weirdo lock.

Hopefully this will once and for all fix the deadlocks with this. With this
commit there are two locks on each mount:

- krwlock_t mnt_unmounting. This is used to prevent unmount across critical
sections like getnewvnode(). It's only ever read locked with rw_tryenter(),
and is only ever write locked in dounmount(). A write hold can't be taken
on this lock if the current LWP could hold a vnode lock.

- kmutex_t mnt_updating. This is taken by threads updating the mount, for
example when going r/o -> r/w, and is only present to serialize updates.
In order to take this lock, a read hold must first be taken on
mnt_unmounting, and the two need to be held across the operation.

One effect of this change: previously if an unmount failed, we would make a
half hearted attempt to back out of it gracefully, but that was unlikely to
work in a lot of cases. Now while an unmount that will be aborted is in
progress, new file operations within the mount will fail instead of being
delayed. That is unlikely to be a problem though, because if the admin
requests unmount of a file system then s(he) has made a decision to deny
access to the resource.

PR kern/38135 vfs_busy/vfs_trybusy confusion

The previous fix worked, but it opened a window where mounts could have
disappeared from mountlist while the caller was traversing it using
vfs_trybusy(). Fix that.

kern/38135 vfs_busy/vfs_trybusy confusion

The symptom was that sometimes file systems would occasionally not appear
in output from 'df' or 'mount' if the system was busy. Resolution:

- Make mount locks work somewhat like vm_map locks.
- vfs_trybusy() now only fails if the mount is gone, or if someone is
unmounting the file system. Simple contention on mnt_lock doesn't
cause it to fail.
- vfs_busy() will wait even if the file system is being unmounted.

branches: 1.30.2; 1.30.4;
Convert to ansi definitions from old-style definitons.

branches: 1.29.2; 1.29.6;
Restore a fragment of a comment that apparently got lost in a merge slipup
in 1998.

PR kern/37706 (forced unmount of file systems is unsafe):

- Do reference counting for 'struct mount'. Each vnode associated with a
mount takes a reference, and in turn the mount takes a reference to the
vfsops.
- Now that mounts are reference counted, replace the overcomplicated mount
locking inherited from 4.4BSD with a recursable rwlock.

Remove VOP_LEASE. Discussed on tech-kern.

union_putpages: fix a locking botch.

Merge vmlocking2 to head.

branches: 1.24.4;
Remove cn_lwp from struct componentname. curlwp should be used
from on. The NDINIT() macro no longer takes the lwp parameter and
associates the credentials of the calling thread with the namei
structure.

branches: 1.23.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.22.4;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

branches: 1.21.4; 1.21.6; 1.21.8; 1.21.10;
apply ketchup to vop_mmap_args comment

branches: 1.20.2;
define a pager flag PGO_RECLAIM, similar to FSYNC_RECLAIM, and use it
to skip unnecessary flushing when layered file system vnodes are recycled.
this also prevents a deadlock with the dodgy LFS putpages routine.
fixes the non-LFS part of PR 36150.

branches: 1.19.2; 1.19.6; 1.19.8;
more fixes for the new vnode locking scheme:
- don't use SAVESTART in calls to relookup() from unionfs,
just vref() the desired vnode when we need to.
- fix locking and refcounting in the unionfs EEXIST error cases.
- release any vnode locks before calling VFS_ROOT(), vfs_busy() is enough.
this allows us to simplify union_root() and fix PR 3006.
- union_lock() doesn't handle shared lock requests correctly,
so convert them to exclusive instead. fixes PR 34775.
- in relookup(), avoid reusing "dp" for different purposes,
the error handling wasn't right. (actually just get rid of dp.)
also, change relookup() to ignore LOCKLEAF and always return the
vnode locked since the callers already expect this.

a smorgasbord of improvements to vnode locking and path lookup:
- LOCKPARENT is no longer relevant for lookup(), relookup() or VOP_LOOKUP().
these now always return the parent vnode locked. namei() works as before.
lookup() and various other paths no longer acquire vnode locks in the
wrong order via vrele(). fixes PR 32535.
as a nice side effect, path lookup is also up to 25% faster.
- the above allows us to get rid of PDIRUNLOCK.
- also get rid of WANTPARENT (just use LOCKPARENT and unlock it).
- remove an assumption in layer_node_find() that all file systems implement
a recursive VOP_LOCK() (unionfs doesn't).
- require that all file systems supply vfs_vptofh and vfs_fhtovp routines.
fill in eopnotsupp() for file systems that don't support being exported
and remove the checks for NULL. (layerfs calls these without checking.)
- in union_lookup1(), don't change refcounts in the ISDOTDOT case, just
adjust which vnode is locked. fixes PR 33374.
- apply fixes for ufs_rename() from ufs_vnops.c rev. 1.61 to ext2fs_rename().

branches: 1.17.2;
Coverity CID 2960: Don't print NULL vnode (from Arnaud Lacombe)

branches: 1.16.8; 1.16.10;
integrate kauth.

Coverity CID 1001: Avoid NULL deref.

branches: 1.14.4; 1.14.6; 1.14.8; 1.14.10; 1.14.12;
merge ktrace-lwp.

branches: 1.13.2;
merge yamt-vop branch. remove following VOPs.

VOP_BLKATOFF
VOP_VALLOC
VOP_BALLOC
VOP_REALLOCBLKS
VOP_VFREE
VOP_TRUNCATE
VOP_UPDATE

branches: 1.12.2;
Remove __P()

branches: 1.11.4;
nuke trailing whitespace

branches: 1.10.4; 1.10.6;
First pass for some caddr_t removal and changes to get rid of it where we
no longer use and/or need it

- removed casts from unionfs, deadfs and fdesc
(there are more to hunt down still)
- changed vfs_quotactl args argumet from caddr_t to void *
- changed vfs_quotactl structures/callers to reflect the api change

Compiled fine and ran for about a day. Approved/reviewed by
christos@netbsd.org and gimpy@netbsd.org.

Replace the statfs() family of system calls with statvfs().
Retain binary compatibility.

Make VOP_STRATEGY(bp) a real VOP as discussed on tech-kern.

VOP_STRATEGY(bp) is replaced by one of two new functions:

- VOP_STRATEGY(vp, bp) Call the strategy routine of vp for bp.
- DEV_STRATEGY(bp) Call the d_strategy routine of bp->b_dev for bp.

DEV_STRATEGY(bp) is used only for block-to-block device situations.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.6.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Undo part of the ktrace/lwp changes. In particular:
* Remove the "lwp *" argument that was added to vget(). Turns out
that nothing actually used it!
* Remove the "lwp *" arguments that were added to VFS_ROOT(), VFS_VGET(),
and VFS_FHTOVP(); all they did was pass it to vget() (which, as noted
above, didn't use it).
* Remove all of the "lwp *" arguments to internal functions that were added
just to appease the above.

Fix problems with Darren's ktrace/lwp changes.

More changes for providing lwpid for ktrace (sparc GENERIC built)

add kqueue support

move union filesystem code from sys/miscfs/union to sys/fs/union

Overhaul of the EVFILT_VNODE kevent(2) filter:

- Centralize vnode kevent handling in the VOP_*() wrappers, rather than
forcing each individual file system to deal with it (except VOP_RENAME(),
because VOP_RENAME() is a mess and we currently have 2 different ways
of handling it; at least it's reasonably well-centralized in the "new"
way).
- Add support for NOTE_OPEN, NOTE_CLOSE, NOTE_CLOSE_WRITE, and NOTE_READ,
compatible with the same events in FreeBSD.
- Track which kevent notifications clients are interested in receiving
to avoid doing work for events no one cares about (avoiding, e.g.
taking locks and traversing the klist to send a NOTE_WRITE when
someone is merely watching for a file to be deleted, for example).

In support of the above:

- Add support in vnode_if.sh for specifying PRE- and POST-op handlers,
to be invoked before and after vop_pre() and vop_post(), respectively.
Basic idea from FreeBSD, but implemented differently.
- Add support in vnode_if.sh for specifying CONTEXT fields in the
vop_*_args structures. These context fields are used to convey information
between the file system VOP function and the VOP wrapper, but do not
occupy an argument slot in the VOP_*() call itself. These context fields
are initialized and subsequently interpreted by PRE- and POST-op handlers.
- Version VOP_REMOVE(), uses the a context field for the file system to report
back the resulting link count of the target vnode. Return this in tmpfs,
udf, nfs, chfs, ext2fs, lfs, and ufs.

NetBSD 9.99.92.

Fix union_parsepath(), either the upper or the lower dvp may be NULL.

Now remove cn_consume from struct componentname.

This change requires a kernel bump.

Note though that I'm not going to version the VOP_LOOKUP args
structure (or any other args structure) as code that doesn't touch
cn_consume doesn't need attention and code that does will fail on it
without further intervention.

Onionfs needs to know about parsepath too, in case it has one of the
other cases underneath it.

The solution here is not really very good (take the longer
path-to-consume if they're different) but it will serve for the cases
that exist.

(If we were to add a fs that really uses different naming semantics,
we'd have to take additional steps; probably it doesn't make sense to
allow unionfs to union such a thing with a normal fs and attempting it
should fail at mount time.)

Update fs/unionfs as well to avoid increasing the current set of
compile failures there. Though maybe it's time to just remove
fs/unionfs.

- Add a new vnode op: VOP_PARSEPATH.
- Move namei_getcomponent to genfs_vnops.c and call it genfs_parsepath.
- Add a parsepath entry to every vnode ops table.

VOP_PARSEPATH takes a directory vnode to be searched and a complete
following path and chooses how much of that path to consume. To begin
with, all parsepath calls are genfs_parsepath, which locates the first
'/' as always.

Note that the call doesn't take the whole struct componentname, only
the string. The other bits of struct componentname should not be
needed and there's no reason to cause potential complications by
exposing them.

branches: 1.74.6;
Operation union_readdirhook() stores the lower directory as un_uppervp.
This breaks the assumption that un_uppervp->v_mount is the upper mount.

Fix by storing the directory as un_lowervp and adapt union_readdir().

Should fix PR kern/55552: panic with union mount

Add ACL support for FFS. From FreeBSD.

UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.70.10; 1.70.14; 1.70.16;
Make VOP_RECLAIM do the last unlock of the vnode.

VOP_RECLAIM naturally has exclusive access to the vnode, so having it
locked on entry is not strictly necessary -- but it means if there
are any final operations that must be done on the vnode, such as
ffs_update, requiring exclusive access to it, we can now kassert that
the vnode is locked in those operations.

We can't just have the caller release the last lock because some file
systems don't use genfs_lock, and require the vnode to remain valid
for VOP_UNLOCK to work, notably unionfs.

Use VCALL() to lock or unlock the lower node.

Move v_writecount adjustment from revoke to reclaim.

branches: 1.67.2;
Change VOP_REMOVE and VOP_RMDIR to preserve lock/ref on dvp.

No change to vp -- the plan is to replace the node by the
componentname in the vop parameters, and let all directory vops do
lookups internally.

Proposed on tech-kern with no objections:
https://mail-index.netbsd.org/tech-kern/2017/04/17/msg021825.html

Remove unused argument "nextp" from vfs_busy() and vfs_unbusy().
Remove argument "keepref" from vfs_unbusy() and add vfs_ref() where needed.

Make VOP_INACTIVE preserve vnode lock on return.

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2017/04/01/msg021751.html

Ride 7.99.68, a bumpy bus of incremental vfs improvements!

Handle v_writecount from union_open(), union_close() and union_revoke()
so lower file system vnodes get marked as open for writing.

branches: 1.63.2; 1.63.4;
Make VOP_LINK return directory still locked and referenced.

Ride 7.99.10 bump.

branches: 1.62.4;
Add VOP_FALLOCATE and VOP_FDISCARD to every vnode ops table I can
find.

The filesystem ones all call genfs_eopnotsupp - right now I am only
implementing the plumbing and we can implement fallocate and/or
fdiscard for files later.

The device ones call spec_fallocate (which is also genfs_eopnotsupp)
and spec_fdiscard, which dispatches to the device-level op.

The fifo ones all call vn_fifo_bypass, which also ends up being
EOPNOTSUPP.

Also set or assert that *vpp is null before calling VOP_MKDIR.

Set *vpp to NULL before calling VOP_CREATE. This always happens when
calling using nameidata, and if not something went wrong, so we'd like
to be able to assert about it.

branches: 1.59.2;
- Make VI_XLOCK, VI_CLEAN and VI_LOCKSHARE private to kern/vfs_*.c.
- Make vwait() static.
- Add vdead_check() to check a vnode for being or becoming dead.

Discussed on tech-kern.

Welcome to 6.99.38

Restructure union_lock() to always lock before testing for dead node.
Add two little helpers to lock or unlock a node. Use "vp" for the
union node and "lockvp" for the node to be locked. Use ISSET() to
test flags, add assertions.

The current implementation of vn_lock() is racy. Modification of
the vnode operations vector for active vnodes is unsafe because it
is not known whether deadfs or the original file system will be
called.

- Pass down LK_RETRY to the lock operation (hint for deadfs only).

- Change deadfs lock operation to return ENOENT if LK_RETRY is unset.

- Change all other lock operations to check for dead vnode once
the vnode is locked and unlock and return ENOENT in this case.

With these changes in place vnode lock operations will never succeed
after vclean() has marked the vnode as VI_XLOCK and before vclean()
has changed the operations vector.

Adresses PR kern/37706 (Forced unmount of file systems is unsafe)

Discussed on tech-kern.

Welcome to 6.99.33

Change union_allocvp() to take an unlocked uppervp and to return the
union node unlocked. Another VI_XLOCK hack is gone.

Remove an unused variable

Get rid of UN_KLOCK to keep a lock on vput(). It is not really needed
and makes the source difficult to read. Always hold references to the
union nodes until the operation is done.

Fix the DOT and DOTDOT case for union_lookup1().

Change vnode operation lookup to return the resulting vnode *vpp unlocked.
Change cache_lookup() to return an unlocked vnode.

Discussed on tech-kern@

Welcome to 6.99.31

Change vnode operations create, mknod, mkdir and symlink to return
the resulting vnode *vpp unlocked.

Discussed on tech-kern@

Welcome to 6.99.30

Change vnode operations create, mknod, mkdir and symlink to keep the
directory node dvp locked on return.

Discussed on tech-kern@

Welcome to 6.99.29

branches: 1.49.8; 1.49.12;
Replace flag based union node locking with generic vnode lock, support
shared and nowait locks and protect un_uppervp and un_*sz with mutex.

Mark file system MPSAFE.

VOP_ABORTOP() has no specific lock requirements so there is no need
to force locked vnodes here. It should be impossible to come here
with a nil upper node.

Relock the directory vnode after copyup. A locked union node with an
unlocked upper vnode can no longer exist so make FIXUP() an assertion.

branches: 1.47.2;
VOP_GETATTR() needs a shared lock at least.

Stop abusing relookup() to prepare the creation of new nodes
in the upper layer.
Replace union_relookup() with union_do_lookup() that prepares
a component, calls VOP_LOOKUP() and does the EEXIST test.

Change some `#ifdef DIAGNOSTIC' to `KASSERT'.
Instead of a `pid_t' use a `lwp_t *' for locking diagnostics.

No functional changes intended.

Add missing parts to mount devices from a union file system:
- union_close() has to lock/unlock the lower vnode.
- union_fsync() has to call spec_fsync() for the union vnode.
- union_strategy() must allow writes to devices on the lower file system.
- union_bwrite() was completely missing.

For devices, sockets and fifos ignore setting the file size to zero to make
open(..., O_TRUNC) happy and allow them to write through the lower layer.

Fixes PR #43560 (writing to null device in unionfs fails)

Change union rmdir semantics to fail directory removal for
non-empty directories like all other file systems do.

Change test accordingly.

When union_lookup() creates a shadow directory and nameiop is not LOOKUP
it has to restart the lookup to get the componentname right.

Fixes PR #44383 (an endless stream of whiteout and opaque dir problems ...)