ffs: apply the remaining ffs_snapshot.c part of this FreeBSD commit:

commit 364ed814e7285c8216d8a201d3ab3674eb34ce29
Author: Kirk McKusick <mckusick@FreeBSD.org>
Date: Thu Dec 9 21:24:00 2004 +0000

Fixes a bug that caused UFS2 filesystems bigger than 2TB to
prematurely report that they were full and/or to panic the kernel
with the message ``ffs_clusteralloc: allocated out of group''.

Submitted by: Henry Whincup <henry@jot.to>
MFC after: 1 week

all the other changes in that commit were applied previously by others:
- sborrill commmitted ffs_alloc.c rev 1.123 in 2009
- simonb committed ffs_alloc.c rev 1.110 in 2008
- the ffs_clusteralloc() part is not needed because we no longer have
that function.

fixes PR 57307

branches: 1.154.4;
Take the link count from the inode.

s/shapshot/snapshot/

Extended attribute support for ffsv2, from FreeBSD.

branches: 1.151.4;
UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.149.10; 1.149.14; 1.149.16;
remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.

KASSERT(mutex_owned(vp->v_interlock)) in vnode iterator selector.

#if DIAGNOSTIC panic ---> KASSERT

Remove now redundant calls to fstrans_start()/fstrans_done().

Bring back vrele_flush() to flush deferred vrele() o an suspended file system.

Untangle VFS_SYNC() from VFS_SUSPENDCTL().

branches: 1.143.2;
reorganize ffs_truncate()/ffs_indirtrunc() to be able to partially
succeed; change wapbl_register_deallocation() to return EAGAIN
rather than panic when code hits the limit

callers changed to either loop calling ffs_truncate() using new
utility ufs_truncate_retry() if their semantics requires it, or
just ignore the failure; remove ufs_wapbl_truncate()

this fixes possible user-triggerable panic during truncate, and
resolves WAPBL performance issue with truncates of large files

PR kern/47146 and kern/49175

revert 1.141 - the second ffs_truncate() can't really fail

requested by hannken@

allow also the snapshot_setup()'s call to ffs_truncate() fail, the code
should simply reuse the file blocks in that case; also make sure the
ffs_truncate() call is run within transaction if log is on

branches: 1.140.2;
Small fixes.

ok hannken@

Remove the 'cred' argument from bread(). Remove a now unused var in
ffs_snapshot.c. Update the man page accordingly.

ok hannken@

Remove the 'cred' argument from breadn(), and update the man page
accordingly.

ok hannken@

branches: 1.137.2;
Don't nest structure definitions.

Use an explicit compare to 0 for an immediate error result, not !.
Using ! is perfectly clear on variables like "error" or "result",
but directly on a function call it tends to look like a mistake.

Testing "v_usecount == 1" for exclusive reference will not always
work -- remove and test only readonly.

Introduce a selector function to the vfs vnode iterator so that we don't
need to vget() vnodes that we are not interested at, and optimize locking
a bit. Iterator changes reviewed by Hannken (thanks), the rest of the bugs
are mine.

branches: 1.133.2;
Change snapshot_expunge() to use vfs_vnode_iterator.

ib_get is not used in the evbarm/OPENRD kernel, so mark it as such.

Mark unused (in the !FFS_EI case) variables as such.

Mark a potentially unused (ifndef FFS_EI) variable

Replace macro v_specmountpoint with two functions spec_node_getmountedfs()
and spec_node_setmountedfs() to manage the file system mounted on a device.
Assert the device is a block device.

Welcome to 6.99.24

Discussed on tech-kern@ some time ago.

Reviewed by: David Holland <dholland@netbsd.org>

Kill unused function ib_assign.

branches: 1.127.2;
Stick ffs_ in front of the following macros:
fragstoblks()
blkstofrags()
fragnum()
blknum()

to finish the job of distinguishing them from the lfs versions, which
Christos renamed the other day.

I believe this is the last of the overtly ambiguous exported symbols
from ffs... or at least, the last of the ones that conflicted with lfs.
ffs still pollutes the C namespace very broadly (as does ufs) and this
needs quite a bit more cleanup.

XXX: boo on macros with lowercase names. But I'm not tackling that just yet.

Stick ffs_, ext2_, chfs_, filecore_, cd9660_, or mfs_ in front of
the following symbols so as to disambiguate fully. (Christos already
did the lfs ones.)

lblkno
lblktosize
lfragtosize
numfrags
blkroundup
fragroundup

fsbtodb() -> FFS_FSBTODB(), EXT2_FSBTODB(), or MFS_FSBTODB()
dbtofsb() -> FFS_DBTOFSB() or EXT2_DBTOFSB()

(Christos already did the lfs ones a few days back)

Rename ambiguous macros:
MAXDIRSIZE -> UFS_MAXDIRSIZE or LFS_MAXDIRSIZE
NINDIR -> FFS_NINDIR, EXT2_NINDIR, LFS_NINDIR, or MFS_NINDIR
INOPB -> FFS_INOPB, LFS_INOPB
INOPF -> FFS_INOPF, LFS_INOPF
blksize -> ffs_blksize, ext2_blksize, or lfs_blksize
sblksize -> ffs_blksize

These are not the only ambiguously defined filesystem macros, of
course, there's a pile more. I may not have found all the ambiguous
definitions of blksize(), too, as there are a lot of other things
called 'blksize' in the system.

Add an UFS_SNAPGONE() ufs op replacing the calls
to ffs_snapgone() in ufs_lookup.c.

Ok: David Holland <dholland@netbsd.org>

Welcome to 6.99.22

When invalidating short buffers on the snapshots clean list use bbusy()
to mark the buffer busy. There exists a small window where a buffer is
done but not released and therefore still busy.

Stuff UFS_ in front of a few of ufs's symbols to reduce namespace
pollution. Specifically:
ROOTINO -> UFS_ROOTINO
WINO -> UFS_WINO
NXADDR -> UFS_NXADDR
NDADDR -> UFS_NDADDR
NIADDR -> UFS_NIADDR
MAXSYMLINKLEN -> UFS_MAXSYMLINKLEN
MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Sort out ext2fs's misuse of NDADDR and NIADDR; fortunately, these have
the same values in ext2fs and ffs.

No functional change intended.

Change bread() and breadn() to never return a buffer on
error and modify all callers to not brelse() on error.

Welcome to 6.99.16

PR kern/46282 (6.0_BETA crash: msdosfs_bmap -> pcbmap -> bread -> bio_doread)

branches: 1.119.2;
Replace the remaining KAUTH_GENERIC_ISSUSER authorization calls with
something meaningful. All relevant documentation has been updated or
written.

Most of these changes were brought up in the following messages:

http://mail-index.netbsd.org/tech-kern/2012/01/18/msg012490.html
http://mail-index.netbsd.org/tech-kern/2012/01/19/msg012502.html
http://mail-index.netbsd.org/tech-kern/2012/02/17/msg012728.html

Thanks to christos, manu, njoly, and jmmv for input.

Huge thanks to pgoyette for spinning these changes through some build
cycles and ATF.

branches: 1.118.2; 1.118.6;
As vnalloc() always allocates with PR_WAITOK there is no longer the need
to test its result for NULL.

ffs_copyonwrite(): If the write is to the in-file-system journal
there is no need to lock and check the snapshots.

Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

branches: 1.115.2;
Revert previous commit. Locking the snapshot vnode while the file system
is suspended extends the suspension until the vnode gets unlocked by
the caller of ffs_snapshot().

Resuming the file system before expunging all snapshots and syncing the
snapshot creates races and deadlocks with journaling file systems at least.

Before expunging all snapshots take the snapshot lock and resume the file
system as this is sufficient for the remaining operations.

Reduces the time the file system is suspended and should make this time
independent of the number of snapshots already present.

ffs_snapshot(): return an error if the node is an invalid snapshot.

Preallocate all cylinder group blocks so we no longer redo ~50% of
the cylinder groups while the file system is suspended.
This was removed in error with Rev 1.16.

From Manuel Bouyer <bouyer@netbsd.org> via tech-kern.

merge the bouyer-quota2 branch. This adds a new on-disk format
to store disk quota usage and limits, integrated with ffs
metadata. Usage is checked by fsck_ffs (no more quotacheck)
and is covered by the WAPBL journal. Enabled with kernel
option QUOTA2 (added where QUOTA was enabled in kernel config files),
turned on with tunefs(8) on a per-filesystem
basis. mount_mfs(8) can also turn quotas on.

See http://mail-index.netbsd.org/tech-kern/2011/02/19/msg010025.html
for details.

fss(4): Allow FSSIOCSET to set the initial flags. Add a new flag
"FSS_UNLINK_ON_CREATE" to unlink the backing store before
the snapshot gets created.

With this change dump(8) no longer dumps the zero-sized, but named
snapshot it is working on. Same applies to fsck_ffs(8).

Initialize blkno to 0 right before the snapblkaddr() call that GCC does
not understand so that if ffs_copyonwrite() sprouts a new code path that
does not initialize blkno, the compiler has the chance to reveal it.

Quiesce CC ('blkno' may be used uninitialized in this function).

Move blocks_in_journal() in under #ifndef FFS_NO_SNAPSHOT, all uses
are under that ifdef anyway; this allows build with FFS_NO_SNAPSHOT defined.

Change the snapshot lock:
- No need to take the snapshot lock while the file system is suspended.
- Allow ffs_copyonwrite() one level of recursion with snapshots locked.
- Do the block address lookup with snapshots locked.
- Take the snapshot lock while removing a snapshot from the list.

While hunting deadlocks change the transaction scope for ffs_snapremove().
We could deadlock from UFS_WAPBL_BEGIN() with a buffer held.

Initialize error in snapshot_expunge(); if the list is empty error would
be returned uninitialized. t_snapshot_v2 was failing for me when
librumpffs was compiled DGB=-g.
No idea why gcc didn't catch this ...

Revert rev. 1.101. Dead snapshots would hang around until unmount.

Adresses PR #44568 (WAPBL doens't play nice with snapshots).

Refine the scope of WAPBL transactions so we should no longer get
a "wapbl_flush: current transaction too big to flush" panic when
creating or removing snapshots on larger logging disks.

Adresses PR #44568 (WAPBL doens't play nice with snapshots).

branches: 1.102.2; 1.102.4;
Move counting of faults, traps, intrs, soft[intr]s, syscalls, and nswtch
from uvmexp to per-cpu cpu_data and move them to 64bits. Remove unneeded
includes of <uvm/uvm_extern.h> and/or <uvm/uvm.h>.

Keep a reference to the snapshot vnode until it gets removed from the
snapshot list.

syncsnap: Use bbusy() to take a buffer from v_dirtyblkhd.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

Initialize the initial snap block list's count.

From Antti Kantee <pooka@netbsd.org>.

branches: 1.97.2; 1.97.4;
No longer abuse TAILQ internal data.

Fix a deadlock where fscow_disestablish() blocks because outstanding
copy-on-write operations wait for si_snaplock.

Remove extra whitespace added by a stupid tool.
XXX: more in src/sys/arch

bcopy -> memcpy

bzero -> memset

PR kern/26878 FFSv2 + softdep = livelock (no free ram)
PR kern/16942 panic with softdep and quotas
PR kern/19565 panic: softdep_write_inodeblock: indirect pointer #1 mismatch
PR kern/26274 softdep panic: allocdirect_merge: ...
PR kern/26374 Long delay before non-root users can write to softdep partitions
PR kern/28621 1.6.x "vp != NULL" panic in ffs_softdep.c:4653 while unmounting a softdep (+quota) filesystem
PR kern/29513 FFS+Softdep panic with unfsck-able file-corruption
PR kern/31544 The ffs softdep code appears to fail to write dirty bits to disk
PR kern/31981 stopping scsi disk can cause panic (softdep)
PR kern/32116 kernel panic in softdep (assertion failure)
PR kern/32532 softdep_trackbufs deadlock
PR kern/37191 softdep: locking against myself
PR kern/40474 Kernel panic after remounting raid root with softdep

Retire softdep, pass 2. As discussed and later formally announced on the
mailing lists.

branches: 1.91.2;
merge christos-time_t

Remove superfluous "vp->v_vnlock = &vp->v_lock".

Observed by: YAMAMOTO Takashi <yamt@netbsd.org>

Restore a line removed by mistake with the last commit.

Should fix PR 40225 panic: indiracct: missing indir.

kill MALLOC and FREE macros.

ffs_copyonwrite(): Only use si_snapblklist if it is already allocated.

ffs_snapshot_read(): Use IO_ALTSEMANTICS to allow reading a snapshot vnode
beyond file system size. Needed to read the snapblklist
on mount.

Persistent snapshots work again.

Should fix PR kern/37425: fss_snapshot_mount panic during fsck.

Revert previous -- ALL reads are from kernel space.

Still open: PR kern/37425: fss_snapshot_mount panic during fsck.

ffs_copyonwrite(): Only use si_snapblklist if it is already allocated.
ffs_snapshot_read(): Allow the kernel to read beyond file system size.

Persistent snapshots work again.

Should fix PR kern/37425: fss_snapshot_mount panic during fsck.

Split ffs_freefile into a frontend for normal cylinder group and for
snapshot use. Adjust ffs_blkfree_common to get the fs instance passed
in, the original commit didn't account blocks in the snapshots
correctly. Assert that ffs_blkfree is used with the primary fs instance
and that ffs_checkfreefile is only used for snapshots. Move the bdwrite
from ffs_blkfree_common into the caller for symmetry. This creates a
redundant write of unmodified data for ffs_blkfree_snap if a double free
of a block happens.

Reviewed and tested by hannken@.

ffs_blkfree is used in two different ways. The normal usage is to free a
block in the cylinder groups of the filesystem. The other user is the
snapshot code, which wants to modify the copied cylinder groups. Use
different frontends to distinguish the cases in preparation for fine
grained locking for cylinder groups.

branches: 1.82.2; 1.82.4;
Correct previous.
- Count frags, not blocks to get the file system size.
- Cannot use blksize() here, it depends on vnode size.
- Correctly update xfersize on short reads.

When computing the requests hard limit in ffs_snapshot_read()
use the file system size, not the size of the snapshot vnode.

Adjust some WAPBL transactions:
- Put transaction inside cgaccount() to simplify caller.
- No vget() / vrele() inside a transaction.

Ffs_snapshot() has become a huge monster over the time. Break it into
helper functions to enhance readability. Adjust comments to reality
and test the main error paths.

While here, expand and remove the last FreeBSD->NetBSD conversion macros.

No functional change intended.

Sync the just created snapshot to disk.

Invalidate short ( < fs_bsize ) buffers. We will always read full
size buffers later.

Should fix PR #39402

Add missing vput() for logvp.

Fixes PR #39400

Merge the _ufs1 and _ufs2 variants of the expunge and accounting functions.
Remove some unneeded UFS_FSNEEDSWAP().

Saves ~250 lines of redundant code.

Add snapshot support for logging ffs file systems.

- Add UFS_WAPBL_BEGIN() / UFS_WAPBL_END() where needed.

- Expunge WAPBL log inodes from snapshots.

- Ffs_copyonwrite() and ffs_snapblkfree() must run inside a WAPBL transaction.

- Add ffs_gop_write() as a wrapper around genfs_gop_write() that makes sure
genfs_gop_write() gets always called inside a WAPBL transaction.

- Add VOP_PUTPAGES() flag PGO_JOURNALLOCKED to tag calls to VOP_PUTPAGES()
inside a WAPBL transaction.

Reviewed by: Simon Burge <simonb@netbsd.org>, Greg Oster <oster@netbsd.org>

PGO_JOURNALLOCKED / ffs_gop_write() part presented on tech-kern@.

Deny read/write access to snapshot vnodes. We use fss(4) to read from
snapshots. With this policy in place:

- Separate the snapshot vnode lock from the snapshot common lock.
Snapshots no longer need recursive vnode locks.

- Use a mutex (si_snaplock) to serialize creation, deletion, reading and
writing of snapshots.

- Move ffs_read() for snapshots into ffs_snapshot.c.

Reviewed by: Jason Thorpe <thorpej@netbsd.org>

While here change ffs_copyonwrite() to fail requests from pagedaemon that need
to copy-on-write.

Ffs snapshots don't work (yet) with WAPBL:
- no snapshot creation on logging file systems.
- refuse to mount logging file systems with persistent snapshots.

Ok: Simon Burge <simonb@netbsd.org>

ffs_snapshot():
Release allocated indir blocks on non-softdep file systems instead
of writing them twice.
It is sufficient to clean dirty data pages to avoid UBC inconsistencies.

ffs_snapblkfree() and wrsnapblk():
If a snapshots effective link count is zero there is no need
to use synchronous writes.

ffs_copyonwrite():
Defer locking the snapshots until there is a need to copy the block.

wrsnapblk():
Use vn_rdwr() instead of bwrite() to write to the snapshots.

expunge_ufs*(): Use the buffer cache to update the inodes on the snapshot like
the rest of snapshot creation does.

branches: 1.70.2;
Mark a buffer `busy` in getnewbuf() when it came from the pool_cache since
its not on a free list.

Also change buf_init() to not automatically mark buffers `busy' since this
only makes sense for bufcache buffers.

Mark all buf_init'd buffers 'busy' on the places where they ought to be
flagged as such to not confuse the buffer cache.

Fixes PR 38923.

branches: 1.69.2;
ufs/ffs: replace calls to getblk() with ffs_getblk(). Now all buffers
have been run through copy-on-write and async mounts work again.

Fixes PR kern/38820

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

ffs_copyonwrite(): stop abusing ffs_balloc() to get a block address.
Use ufs_getlbns()/bread() instead.
Saves some reads and removes deep recursion with possible deadlock
when ffs_balloc() runs copy-on-write on the buffer returned.

Make sure all cached buffers with valid, not yet written data have been
run through copy-on-write. Call fscow_run() with valid data where possible.

The LP_UFSCOW hack is no longer needed to protect ffs_copyonwrite() against
endless recursion.

- Add a flag B_MODIFY to bread(), breada() and breadn(). If set the caller
intends to modify the buffer returned.

- Always run copy-on-write on buffers returned from ffs_balloc().

- Add new function ffs_getblk() that gets a buffer, assigns a new blkno,
may clear the buffer and runs copy-on-write. Process possible errors
from getblk() or fscow_run(). Part of PR kern/38664.

Welcome to 4.99.63

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

branches: 1.66.2; 1.66.4; 1.66.6;
Replace get/setspecific with a void pointer in struct ufsmount. Use explicit
initialization/finalization of snapshot private data on creation/deletion
of struct ufsmount.
Snapshot mounts no longer may fail silently because kmem_alloc() fails.

Welcome to 4.99.60

Ok: Andrew Doran <ad@netbsd.org>

branches: 1.65.6; 1.65.8;
Make it work after lockmgr -> vlockmgr conversion:

- Initialize si_vnlock in si_mount_init().
- Also initialize vl_recursecnt to zero.
- Destroy it only in si_mount_dtor().
- Simplify the v_lock <-> si_vnlock exchange.
- Don't abuse the overall error variable for LK_NOWAIT errors.
- ffs_snapremove: release the vnode one instead of three times.

Replace use of LK_SLEEPFAIL.

PR kern/37706 (forced unmount of file systems is unsafe):

- Do reference counting for 'struct mount'. Each vnode associated with a
mount takes a reference, and in turn the mount takes a reference to the
vfsops.
- Now that mounts are reference counted, replace the overcomplicated mount
locking inherited from 4.4BSD with a recursable rwlock.

Replace struct lock on vnodes with a simpler lock object built on
krwlock_t. This is a step towards removing lockmgr and simplifying
vnode locking. Discussed on tech-kern.

- Always destroy si_vnlock after use.
- Take care of vnodes without file system data.

si_mount_dtor(): destroy si_vnlock before free.

Fix a typo from the vmlocking2 merge: vmark() the right vnode.

valloc -> vnalloc, vfree -> vnfree
Avoids collision with userland valloc(3).

no functional change
ad ok

Merge vmlocking2 to head.

branches: 1.56.4;
Remove cn_lwp from struct componentname. curlwp should be used
from on. The NDINIT() macro no longer takes the lwp parameter and
associates the credentials of the calling thread with the namei
structure.

branches: 1.55.2;
Fscow_run(): add a flag "bool data_valid" to note still valid data.
Buffers run through copy-on-write are marked B_COWDONE. This condition
is valid until the buffer has run through bwrite() and gets cleared from
biodone().

Welcome to 4.99.39.

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.53.4;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

Merge ffs locking & brelse changes from the vmlocking branch.

Update the file system copy-on-write handler.

- Instead of hooking the handler on the specdev of a mounted file system
hook directly on the `struct mount'.

- Rename from `vn_cow_*' to `fscow_*' and move to `kern/vfs_trans.c'. Use
`mount_*specific' instead of clobbering `struct mount' or `struct specinfo'.

- Replace the hand-made reader/writer lock with a krwlock.

- Keep `vn_cow_*' functions and mark as obsolete.

- Welcome to NetBSD 4.99.32 - `struct specinfo' changed size.

Reviewed by: Jason Thorpe <thorpej@netbsd.org>

branches: 1.50.2; 1.50.4;
Modify ffs_lock() to take care for changed v_vnlock. Snapshots do not need
transferlockers() anymore.

From FreeBSD ffs_vnops.c Rev. 1.159

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

- Use a mutex to protect snapinfo.
- Move the snapshot lock to snapinfo.
- ffs_snapblkfree(),ffs_copyonwrite(): replace lockmgr() with VOP_LOCK().

Expunge traces of unlinked snapshot files when making a new snapshot.

From FreeBSD Rev. 1.123

Move snapshot per-mount data from struct ufsmount to mount specific data.
No functional changes.

Welcome to 4.99.28 (struct ufsmount changed size)

branches: 1.46.2; 1.46.6;
ffs_snapshot_mount: No persistent snapshots on an Apple UFS file system.

From Thor Lancelot Simon <tls@netbsd.org>

Move `struct dquot' and its supporting functions from quota.h to ufs_quota.c.

- Make quota-internal functions static.
- Clean up declarations in quota.h and ufs_extern.h. quota.h now has the
description of quota criterions, on-disk structure, user-kernel interface and
declaration of init/done functions. All ufs quota related function
prototypes go to ufs_extern.h.
- New functions ufsquota_init() and ufsquota_free() create or destroy the
quota fields of `struct inode'.
- chkdq() and chkiq() always update the quota fields of `struct inode' first.
- Only ufs_access() explicitely calls getinoquota().

No objections on tech-kern@

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.43.2; 1.43.4;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.42.2;
Make fstrans(9) the default helper for file system suspension.
Replaces the now obsolete vn_start_write()/vn_finished_write().

Merge newlock2 to head.

New file system suspension API to replace vn_start_write and vn_finished_write.
The suspension helpers are now put into file system specific operations.
This means every file system not supporting these helpers cannot be suspended
and therefore snapshots are no longer possible.

Implemented for file systems of type ffs.

The new API is enabled on a kernel option NEWVNGATE. This option is
not enabled by default in any kernel config.

Presented and discussed on tech-kern with much input from
Bill Studenmund <wrstuden@netbsd.org> and YAMAMOTO Takashi <yamt@netbsd.org>.

Welcome to 4.99.9 (new vfs op vfs_suspendctl).

Consistent usage of KAUTH_GENERIC_ISSUSER.

On snapshot creation be sure the snapshot vnode has valid quota information.

Fixes PR kern/35121

branches: 1.37.2;
ifdef out an unused function if !FFS_NO_SNAPSHOT

__unused removal on arguments; approved by core.

Revisit mnt_vnodelist TAILQ patch. Remove all suspicious TAILQ_FOREACH()
loops where vnodes can get removed or added during the loops. This could
lead to panic's on unmount since nodes are skipped or otherwise
TAILQ_NEXT(0xdeadbeef, ...) was dereferenced.

Replace the LIST structure mp->mnt_vnodelist to a TAILQ structure since all
vnodes were synced and processed backwards. This meant that the last
accessed node was processed first and the earlierst last.

An extra benefit is the removal of the ugly hack from the Berkly days on
LFS.

In the proces, i've also replaced the various variations hand written loops
by the TAILQ_FOREACH() macro's.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

Coverity CID 2949: comment out dead code (from Arnaud Lacombe)

branches: 1.31.4; 1.31.6;
Use the LWP cached credentials where sane.

merge FreeBSD timecounters from branch simonb-timecounters
- struct timeval time is gone
time.tv_sec -> time_second
- struct timeval mono_time is gone
mono_time.tv_sec -> time_uptime
- access to time via
{get,}{micro,nano,bin}time()
get* versions are fast but less precise
- support NTP nanokernel implementation (NTP API 4)
- further reading:
Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf
NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html

branches: 1.29.2;
integrate kauth.

Coverity CID 746: Remove dead code. lbn >= NDADDR is mutually exclusive to
snapshot_locked == 0.

Revert previous; I mixed bpp and *bpp when reading ffs_balloc_ufs1().
ffs_balloc() will always allocate a new buffer or leave it as NULL,
so coverity is wrong here, we're not using a freed argument.

If we brelse ibp, set ibp to NULL, to avoid reusing it later in balloc()
or in our code at the next iteration.
Coverity ID 2706

don't use MALLOC with a non-constant size; use malloc instead.

branches: 1.24.2; 1.24.4; 1.24.6; 1.24.8; 1.24.10;
- add simple functions to allocate/free a buffer for i/o.
- make bufpool static.

branches: 1.23.2;
merge ktrace-lwp.

merge yamt-vop branch. remove following VOPs.

VOP_BLKATOFF
VOP_VALLOC
VOP_BALLOC
VOP_REALLOCBLKS
VOP_VFREE
VOP_TRUNCATE
VOP_UPDATE

branches: 1.21.2;
revert ffs_snapshot.c 1.20 because it's bogus. pointed by Simon Burge.

always use nanotime rather than time.
it's bad to mix nanotime and time because it sometimes
make timestamps go backwards.

64 bit inode changes.

Use ANSI function decls.

branches: 1.17.2;
- sprinkle const
- avoid shadow variables.

- Use an empty snap block list to set the initial file size. Snapshot is
now valid from the beginning. No need to copy the last fs block two times.
- No need to allocate the cylinder group blocks twice.
- cgbuf -> sbbuf

ffs/ffs_alloc.c:
- Add a missing ACTIVECG_CLR().

ffs/ffs_snapshot.c:
- Use async/delayed writes for snapshot creation and sync/uncache these buffers
on end. Reduces the time the file system must be suspended.
- Remove um_snaplistsize. Was a duplicate of um_snapblklist[0].
- Byte swap the list of preallocated blocks on read/write instead of access.
- Always keep this list on ip->i_snapblklist so it may be rolled back when the
newest snapshot gets removed. Fixes a rare snapshot corruption when using
more than one snapshot on a file system.

ufs/ufsmount.h:
- Make TAILQ_LAST() possible on member um_snapshots.
- Remove um_snaplistsize. Was a duplicate of um_snapblklist[0].

Fix last commit. The last block of the file system may have changed
even if the last cylinder group is not modified.

Fix an inconsistency where the last block of the snapshot contains old data.

The last block of the file system is written to the snapshot before the
file system is suspended. If the last cylinder group is modified after
the file system is suspended the last block of the snapshot may contain
old data. So update this block again.

don't assign to non-lvalue. found by gcc4.

branches: 1.11.2;
nuke trailing whitespace

Make `options FFS_NO_SNAPSHOT' only disable snapshot creation
while not trashing existing snapshots.

Approved by: core@

Fss device only checks read access to snapshot vode. On snapshot creation
check we are either super-user or owner of the snapshot vnode.

branches: 1.8.2;
Protect calls to `ffs_*_swap' with `#ifdef FFS_EI'.

branches: 1.7.4;
There's no need to pass a proc value when using UIO_SYSSPACE with
vn_rdwr(9) and uiomove(9).

OK'd by Jason Thorpe

While creating a snapshot inodes must be freed from the
snapshot, not from the file system.
ffs_freefile() needs explicit "fs" and "devvp" arguments.

branches: 1.5.2;
When we expunge an unreferenced file from a snapshot its size may be zero.

- Add flag L_COWINPROGRESS to struct lwp to avoid recursion when
doing copy-on-write.

- Change VFS_SNAPSHOT() to return the snapshot vnode locked.

- Make the IO path for copy-on-write and snapshot-read more lightweight.
Avoids deadlocks where vn_rdwr(...READ...) has a shared lock and needs
to copy-on-write.
Avoids deadlocks/panics where to clean pages the copy-on-write needs
to allocate pages for its VOP_PUTPAGES().

L_COWINPROGRESS part approved by: Jason R. Thorpe <thorpej@netbsd.org>

Once all block address modifications are done invalidate and
free all pages from the snapshot vnode.

Make it compile without option FFS_EI.

Add ffs internal snapshots. Written by Marshall Kirk McKusick for FreeBSD.

- Not enabled by default. Needs kernel option FFS_SNAPSHOT.
- Change parameters of ffs_blkfree.
- Let the copy-on-write functions return an error so spec_strategy
may fail if the copy-on-write fails.
- Change genfs_*lock*() to use vp->v_vnlock instead of &vp->v_lock.
- Add flag B_METAONLY to VOP_BALLOC to return indirect block buffer.
- Add a function ffs_checkfreefile needed for snapshot creation.
- Add special handling of snapshot files:
Snapshots may not be opened for writing and the attributes are read-only.
Use the mtime as the time this snapshot was taken.
Deny mtime updates for snapshot files.
- Add function transferlockers to transfer any waiting processes from
one lock to another.
- Add vfsop VFS_SNAPSHOT to take a snapshot and make it accessible through
a vnode.
- Add snapshot support to ls, fsck_ffs and dump.

Welcome to 2.0F.

Approved by: Jason R. Thorpe <thorpej@netbsd.org>

Take the link count from the inode.

s/shapshot/snapshot/

Extended attribute support for ffsv2, from FreeBSD.

branches: 1.151.4;
UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.149.10; 1.149.16;
remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.

KASSERT(mutex_owned(vp->v_interlock)) in vnode iterator selector.

#if DIAGNOSTIC panic ---> KASSERT

Remove now redundant calls to fstrans_start()/fstrans_done().

Bring back vrele_flush() to flush deferred vrele() o an suspended file system.

Untangle VFS_SYNC() from VFS_SUSPENDCTL().

branches: 1.143.2;
reorganize ffs_truncate()/ffs_indirtrunc() to be able to partially
succeed; change wapbl_register_deallocation() to return EAGAIN
rather than panic when code hits the limit

callers changed to either loop calling ffs_truncate() using new
utility ufs_truncate_retry() if their semantics requires it, or
just ignore the failure; remove ufs_wapbl_truncate()

this fixes possible user-triggerable panic during truncate, and
resolves WAPBL performance issue with truncates of large files

PR kern/47146 and kern/49175

revert 1.141 - the second ffs_truncate() can't really fail

requested by hannken@

allow also the snapshot_setup()'s call to ffs_truncate() fail, the code
should simply reuse the file blocks in that case; also make sure the
ffs_truncate() call is run within transaction if log is on

branches: 1.140.2;
Small fixes.

ok hannken@

Remove the 'cred' argument from bread(). Remove a now unused var in
ffs_snapshot.c. Update the man page accordingly.

ok hannken@

Remove the 'cred' argument from breadn(), and update the man page
accordingly.

ok hannken@

branches: 1.137.2;
Don't nest structure definitions.

Use an explicit compare to 0 for an immediate error result, not !.
Using ! is perfectly clear on variables like "error" or "result",
but directly on a function call it tends to look like a mistake.

Testing "v_usecount == 1" for exclusive reference will not always
work -- remove and test only readonly.

Introduce a selector function to the vfs vnode iterator so that we don't
need to vget() vnodes that we are not interested at, and optimize locking
a bit. Iterator changes reviewed by Hannken (thanks), the rest of the bugs
are mine.

branches: 1.133.2;
Change snapshot_expunge() to use vfs_vnode_iterator.

ib_get is not used in the evbarm/OPENRD kernel, so mark it as such.

Mark unused (in the !FFS_EI case) variables as such.

Mark a potentially unused (ifndef FFS_EI) variable

Replace macro v_specmountpoint with two functions spec_node_getmountedfs()
and spec_node_setmountedfs() to manage the file system mounted on a device.
Assert the device is a block device.

Welcome to 6.99.24

Discussed on tech-kern@ some time ago.

Reviewed by: David Holland <dholland@netbsd.org>

Kill unused function ib_assign.

branches: 1.127.2;
Stick ffs_ in front of the following macros:
fragstoblks()
blkstofrags()
fragnum()
blknum()

to finish the job of distinguishing them from the lfs versions, which
Christos renamed the other day.

I believe this is the last of the overtly ambiguous exported symbols
from ffs... or at least, the last of the ones that conflicted with lfs.
ffs still pollutes the C namespace very broadly (as does ufs) and this
needs quite a bit more cleanup.

XXX: boo on macros with lowercase names. But I'm not tackling that just yet.

Stick ffs_, ext2_, chfs_, filecore_, cd9660_, or mfs_ in front of
the following symbols so as to disambiguate fully. (Christos already
did the lfs ones.)

lblkno
lblktosize
lfragtosize
numfrags
blkroundup
fragroundup

fsbtodb() -> FFS_FSBTODB(), EXT2_FSBTODB(), or MFS_FSBTODB()
dbtofsb() -> FFS_DBTOFSB() or EXT2_DBTOFSB()

(Christos already did the lfs ones a few days back)

Rename ambiguous macros:
MAXDIRSIZE -> UFS_MAXDIRSIZE or LFS_MAXDIRSIZE
NINDIR -> FFS_NINDIR, EXT2_NINDIR, LFS_NINDIR, or MFS_NINDIR
INOPB -> FFS_INOPB, LFS_INOPB
INOPF -> FFS_INOPF, LFS_INOPF
blksize -> ffs_blksize, ext2_blksize, or lfs_blksize
sblksize -> ffs_blksize

These are not the only ambiguously defined filesystem macros, of
course, there's a pile more. I may not have found all the ambiguous
definitions of blksize(), too, as there are a lot of other things
called 'blksize' in the system.

Add an UFS_SNAPGONE() ufs op replacing the calls
to ffs_snapgone() in ufs_lookup.c.

Ok: David Holland <dholland@netbsd.org>

Welcome to 6.99.22

When invalidating short buffers on the snapshots clean list use bbusy()
to mark the buffer busy. There exists a small window where a buffer is
done but not released and therefore still busy.

Stuff UFS_ in front of a few of ufs's symbols to reduce namespace
pollution. Specifically:
ROOTINO -> UFS_ROOTINO
WINO -> UFS_WINO
NXADDR -> UFS_NXADDR
NDADDR -> UFS_NDADDR
NIADDR -> UFS_NIADDR
MAXSYMLINKLEN -> UFS_MAXSYMLINKLEN
MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Sort out ext2fs's misuse of NDADDR and NIADDR; fortunately, these have
the same values in ext2fs and ffs.

No functional change intended.

Change bread() and breadn() to never return a buffer on
error and modify all callers to not brelse() on error.

Welcome to 6.99.16

PR kern/46282 (6.0_BETA crash: msdosfs_bmap -> pcbmap -> bread -> bio_doread)

branches: 1.119.2;
Replace the remaining KAUTH_GENERIC_ISSUSER authorization calls with
something meaningful. All relevant documentation has been updated or
written.

Most of these changes were brought up in the following messages:

http://mail-index.netbsd.org/tech-kern/2012/01/18/msg012490.html
http://mail-index.netbsd.org/tech-kern/2012/01/19/msg012502.html
http://mail-index.netbsd.org/tech-kern/2012/02/17/msg012728.html

Thanks to christos, manu, njoly, and jmmv for input.

Huge thanks to pgoyette for spinning these changes through some build
cycles and ATF.

branches: 1.118.2; 1.118.6;
As vnalloc() always allocates with PR_WAITOK there is no longer the need
to test its result for NULL.

ffs_copyonwrite(): If the write is to the in-file-system journal
there is no need to lock and check the snapshots.

Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

branches: 1.115.2;
Revert previous commit. Locking the snapshot vnode while the file system
is suspended extends the suspension until the vnode gets unlocked by
the caller of ffs_snapshot().

Resuming the file system before expunging all snapshots and syncing the
snapshot creates races and deadlocks with journaling file systems at least.

Before expunging all snapshots take the snapshot lock and resume the file
system as this is sufficient for the remaining operations.

Reduces the time the file system is suspended and should make this time
independent of the number of snapshots already present.

ffs_snapshot(): return an error if the node is an invalid snapshot.

Preallocate all cylinder group blocks so we no longer redo ~50% of
the cylinder groups while the file system is suspended.
This was removed in error with Rev 1.16.

From Manuel Bouyer <bouyer@netbsd.org> via tech-kern.

merge the bouyer-quota2 branch. This adds a new on-disk format
to store disk quota usage and limits, integrated with ffs
metadata. Usage is checked by fsck_ffs (no more quotacheck)
and is covered by the WAPBL journal. Enabled with kernel
option QUOTA2 (added where QUOTA was enabled in kernel config files),
turned on with tunefs(8) on a per-filesystem
basis. mount_mfs(8) can also turn quotas on.

See http://mail-index.netbsd.org/tech-kern/2011/02/19/msg010025.html
for details.

fss(4): Allow FSSIOCSET to set the initial flags. Add a new flag
"FSS_UNLINK_ON_CREATE" to unlink the backing store before
the snapshot gets created.

With this change dump(8) no longer dumps the zero-sized, but named
snapshot it is working on. Same applies to fsck_ffs(8).

Initialize blkno to 0 right before the snapblkaddr() call that GCC does
not understand so that if ffs_copyonwrite() sprouts a new code path that
does not initialize blkno, the compiler has the chance to reveal it.

Quiesce CC ('blkno' may be used uninitialized in this function).

Move blocks_in_journal() in under #ifndef FFS_NO_SNAPSHOT, all uses
are under that ifdef anyway; this allows build with FFS_NO_SNAPSHOT defined.

Change the snapshot lock:
- No need to take the snapshot lock while the file system is suspended.
- Allow ffs_copyonwrite() one level of recursion with snapshots locked.
- Do the block address lookup with snapshots locked.
- Take the snapshot lock while removing a snapshot from the list.

While hunting deadlocks change the transaction scope for ffs_snapremove().
We could deadlock from UFS_WAPBL_BEGIN() with a buffer held.

Initialize error in snapshot_expunge(); if the list is empty error would
be returned uninitialized. t_snapshot_v2 was failing for me when
librumpffs was compiled DGB=-g.
No idea why gcc didn't catch this ...

Revert rev. 1.101. Dead snapshots would hang around until unmount.

Adresses PR #44568 (WAPBL doens't play nice with snapshots).

Refine the scope of WAPBL transactions so we should no longer get
a "wapbl_flush: current transaction too big to flush" panic when
creating or removing snapshots on larger logging disks.

Adresses PR #44568 (WAPBL doens't play nice with snapshots).

branches: 1.102.2; 1.102.4;
Move counting of faults, traps, intrs, soft[intr]s, syscalls, and nswtch
from uvmexp to per-cpu cpu_data and move them to 64bits. Remove unneeded
includes of <uvm/uvm_extern.h> and/or <uvm/uvm.h>.

Keep a reference to the snapshot vnode until it gets removed from the
snapshot list.

syncsnap: Use bbusy() to take a buffer from v_dirtyblkhd.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

Initialize the initial snap block list's count.

From Antti Kantee <pooka@netbsd.org>.

branches: 1.97.2; 1.97.4;
No longer abuse TAILQ internal data.

Fix a deadlock where fscow_disestablish() blocks because outstanding
copy-on-write operations wait for si_snaplock.

Remove extra whitespace added by a stupid tool.
XXX: more in src/sys/arch

bcopy -> memcpy

bzero -> memset

PR kern/26878 FFSv2 + softdep = livelock (no free ram)
PR kern/16942 panic with softdep and quotas
PR kern/19565 panic: softdep_write_inodeblock: indirect pointer #1 mismatch
PR kern/26274 softdep panic: allocdirect_merge: ...
PR kern/26374 Long delay before non-root users can write to softdep partitions
PR kern/28621 1.6.x "vp != NULL" panic in ffs_softdep.c:4653 while unmounting a softdep (+quota) filesystem
PR kern/29513 FFS+Softdep panic with unfsck-able file-corruption
PR kern/31544 The ffs softdep code appears to fail to write dirty bits to disk
PR kern/31981 stopping scsi disk can cause panic (softdep)
PR kern/32116 kernel panic in softdep (assertion failure)
PR kern/32532 softdep_trackbufs deadlock
PR kern/37191 softdep: locking against myself
PR kern/40474 Kernel panic after remounting raid root with softdep

Retire softdep, pass 2. As discussed and later formally announced on the
mailing lists.

branches: 1.91.2;
merge christos-time_t

Remove superfluous "vp->v_vnlock = &vp->v_lock".

Observed by: YAMAMOTO Takashi <yamt@netbsd.org>

Restore a line removed by mistake with the last commit.

Should fix PR 40225 panic: indiracct: missing indir.

kill MALLOC and FREE macros.

ffs_copyonwrite(): Only use si_snapblklist if it is already allocated.

ffs_snapshot_read(): Use IO_ALTSEMANTICS to allow reading a snapshot vnode
beyond file system size. Needed to read the snapblklist
on mount.

Persistent snapshots work again.

Should fix PR kern/37425: fss_snapshot_mount panic during fsck.

Revert previous -- ALL reads are from kernel space.

Still open: PR kern/37425: fss_snapshot_mount panic during fsck.

ffs_copyonwrite(): Only use si_snapblklist if it is already allocated.
ffs_snapshot_read(): Allow the kernel to read beyond file system size.

Persistent snapshots work again.

Should fix PR kern/37425: fss_snapshot_mount panic during fsck.

Split ffs_freefile into a frontend for normal cylinder group and for
snapshot use. Adjust ffs_blkfree_common to get the fs instance passed
in, the original commit didn't account blocks in the snapshots
correctly. Assert that ffs_blkfree is used with the primary fs instance
and that ffs_checkfreefile is only used for snapshots. Move the bdwrite
from ffs_blkfree_common into the caller for symmetry. This creates a
redundant write of unmodified data for ffs_blkfree_snap if a double free
of a block happens.

Reviewed and tested by hannken@.

ffs_blkfree is used in two different ways. The normal usage is to free a
block in the cylinder groups of the filesystem. The other user is the
snapshot code, which wants to modify the copied cylinder groups. Use
different frontends to distinguish the cases in preparation for fine
grained locking for cylinder groups.

branches: 1.82.2; 1.82.4;
Correct previous.
- Count frags, not blocks to get the file system size.
- Cannot use blksize() here, it depends on vnode size.
- Correctly update xfersize on short reads.

When computing the requests hard limit in ffs_snapshot_read()
use the file system size, not the size of the snapshot vnode.

Adjust some WAPBL transactions:
- Put transaction inside cgaccount() to simplify caller.
- No vget() / vrele() inside a transaction.

Ffs_snapshot() has become a huge monster over the time. Break it into
helper functions to enhance readability. Adjust comments to reality
and test the main error paths.

While here, expand and remove the last FreeBSD->NetBSD conversion macros.

No functional change intended.

Sync the just created snapshot to disk.

Invalidate short ( < fs_bsize ) buffers. We will always read full
size buffers later.

Should fix PR #39402

Add missing vput() for logvp.

Fixes PR #39400

Merge the _ufs1 and _ufs2 variants of the expunge and accounting functions.
Remove some unneeded UFS_FSNEEDSWAP().

Saves ~250 lines of redundant code.

Add snapshot support for logging ffs file systems.

- Add UFS_WAPBL_BEGIN() / UFS_WAPBL_END() where needed.

- Expunge WAPBL log inodes from snapshots.

- Ffs_copyonwrite() and ffs_snapblkfree() must run inside a WAPBL transaction.

- Add ffs_gop_write() as a wrapper around genfs_gop_write() that makes sure
genfs_gop_write() gets always called inside a WAPBL transaction.

- Add VOP_PUTPAGES() flag PGO_JOURNALLOCKED to tag calls to VOP_PUTPAGES()
inside a WAPBL transaction.

Reviewed by: Simon Burge <simonb@netbsd.org>, Greg Oster <oster@netbsd.org>

PGO_JOURNALLOCKED / ffs_gop_write() part presented on tech-kern@.

Deny read/write access to snapshot vnodes. We use fss(4) to read from
snapshots. With this policy in place:

- Separate the snapshot vnode lock from the snapshot common lock.
Snapshots no longer need recursive vnode locks.

- Use a mutex (si_snaplock) to serialize creation, deletion, reading and
writing of snapshots.

- Move ffs_read() for snapshots into ffs_snapshot.c.

Reviewed by: Jason Thorpe <thorpej@netbsd.org>

While here change ffs_copyonwrite() to fail requests from pagedaemon that need
to copy-on-write.

Ffs snapshots don't work (yet) with WAPBL:
- no snapshot creation on logging file systems.
- refuse to mount logging file systems with persistent snapshots.

Ok: Simon Burge <simonb@netbsd.org>

ffs_snapshot():
Release allocated indir blocks on non-softdep file systems instead
of writing them twice.
It is sufficient to clean dirty data pages to avoid UBC inconsistencies.

ffs_snapblkfree() and wrsnapblk():
If a snapshots effective link count is zero there is no need
to use synchronous writes.

ffs_copyonwrite():
Defer locking the snapshots until there is a need to copy the block.

wrsnapblk():
Use vn_rdwr() instead of bwrite() to write to the snapshots.

expunge_ufs*(): Use the buffer cache to update the inodes on the snapshot like
the rest of snapshot creation does.

branches: 1.70.2;
Mark a buffer `busy` in getnewbuf() when it came from the pool_cache since
its not on a free list.

Also change buf_init() to not automatically mark buffers `busy' since this
only makes sense for bufcache buffers.

Mark all buf_init'd buffers 'busy' on the places where they ought to be
flagged as such to not confuse the buffer cache.

Fixes PR 38923.

branches: 1.69.2;
ufs/ffs: replace calls to getblk() with ffs_getblk(). Now all buffers
have been run through copy-on-write and async mounts work again.

Fixes PR kern/38820

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

ffs_copyonwrite(): stop abusing ffs_balloc() to get a block address.
Use ufs_getlbns()/bread() instead.
Saves some reads and removes deep recursion with possible deadlock
when ffs_balloc() runs copy-on-write on the buffer returned.

Make sure all cached buffers with valid, not yet written data have been
run through copy-on-write. Call fscow_run() with valid data where possible.

The LP_UFSCOW hack is no longer needed to protect ffs_copyonwrite() against
endless recursion.

- Add a flag B_MODIFY to bread(), breada() and breadn(). If set the caller
intends to modify the buffer returned.

- Always run copy-on-write on buffers returned from ffs_balloc().

- Add new function ffs_getblk() that gets a buffer, assigns a new blkno,
may clear the buffer and runs copy-on-write. Process possible errors
from getblk() or fscow_run(). Part of PR kern/38664.

Welcome to 4.99.63

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

branches: 1.66.2; 1.66.4; 1.66.6;
Replace get/setspecific with a void pointer in struct ufsmount. Use explicit
initialization/finalization of snapshot private data on creation/deletion
of struct ufsmount.
Snapshot mounts no longer may fail silently because kmem_alloc() fails.

Welcome to 4.99.60

Ok: Andrew Doran <ad@netbsd.org>

branches: 1.65.6; 1.65.8;
Make it work after lockmgr -> vlockmgr conversion:

- Initialize si_vnlock in si_mount_init().
- Also initialize vl_recursecnt to zero.
- Destroy it only in si_mount_dtor().
- Simplify the v_lock <-> si_vnlock exchange.
- Don't abuse the overall error variable for LK_NOWAIT errors.
- ffs_snapremove: release the vnode one instead of three times.

Replace use of LK_SLEEPFAIL.

PR kern/37706 (forced unmount of file systems is unsafe):

- Do reference counting for 'struct mount'. Each vnode associated with a
mount takes a reference, and in turn the mount takes a reference to the
vfsops.
- Now that mounts are reference counted, replace the overcomplicated mount
locking inherited from 4.4BSD with a recursable rwlock.

Replace struct lock on vnodes with a simpler lock object built on
krwlock_t. This is a step towards removing lockmgr and simplifying
vnode locking. Discussed on tech-kern.

- Always destroy si_vnlock after use.
- Take care of vnodes without file system data.

si_mount_dtor(): destroy si_vnlock before free.

Fix a typo from the vmlocking2 merge: vmark() the right vnode.

valloc -> vnalloc, vfree -> vnfree
Avoids collision with userland valloc(3).

no functional change
ad ok

Merge vmlocking2 to head.

branches: 1.56.4;
Remove cn_lwp from struct componentname. curlwp should be used
from on. The NDINIT() macro no longer takes the lwp parameter and
associates the credentials of the calling thread with the namei
structure.

branches: 1.55.2;
Fscow_run(): add a flag "bool data_valid" to note still valid data.
Buffers run through copy-on-write are marked B_COWDONE. This condition
is valid until the buffer has run through bwrite() and gets cleared from
biodone().

Welcome to 4.99.39.

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.53.4;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

Merge ffs locking & brelse changes from the vmlocking branch.

Update the file system copy-on-write handler.

- Instead of hooking the handler on the specdev of a mounted file system
hook directly on the `struct mount'.

- Rename from `vn_cow_*' to `fscow_*' and move to `kern/vfs_trans.c'. Use
`mount_*specific' instead of clobbering `struct mount' or `struct specinfo'.

- Replace the hand-made reader/writer lock with a krwlock.

- Keep `vn_cow_*' functions and mark as obsolete.

- Welcome to NetBSD 4.99.32 - `struct specinfo' changed size.

Reviewed by: Jason Thorpe <thorpej@netbsd.org>

branches: 1.50.2; 1.50.4;
Modify ffs_lock() to take care for changed v_vnlock. Snapshots do not need
transferlockers() anymore.

From FreeBSD ffs_vnops.c Rev. 1.159

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

- Use a mutex to protect snapinfo.
- Move the snapshot lock to snapinfo.
- ffs_snapblkfree(),ffs_copyonwrite(): replace lockmgr() with VOP_LOCK().

Expunge traces of unlinked snapshot files when making a new snapshot.

From FreeBSD Rev. 1.123

Move snapshot per-mount data from struct ufsmount to mount specific data.
No functional changes.

Welcome to 4.99.28 (struct ufsmount changed size)

branches: 1.46.2; 1.46.6;
ffs_snapshot_mount: No persistent snapshots on an Apple UFS file system.

From Thor Lancelot Simon <tls@netbsd.org>

Move `struct dquot' and its supporting functions from quota.h to ufs_quota.c.

- Make quota-internal functions static.
- Clean up declarations in quota.h and ufs_extern.h. quota.h now has the
description of quota criterions, on-disk structure, user-kernel interface and
declaration of init/done functions. All ufs quota related function
prototypes go to ufs_extern.h.
- New functions ufsquota_init() and ufsquota_free() create or destroy the
quota fields of `struct inode'.
- chkdq() and chkiq() always update the quota fields of `struct inode' first.
- Only ufs_access() explicitely calls getinoquota().

No objections on tech-kern@

Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.43.2; 1.43.4;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.42.2;
Make fstrans(9) the default helper for file system suspension.
Replaces the now obsolete vn_start_write()/vn_finished_write().

Merge newlock2 to head.

New file system suspension API to replace vn_start_write and vn_finished_write.
The suspension helpers are now put into file system specific operations.
This means every file system not supporting these helpers cannot be suspended
and therefore snapshots are no longer possible.

Implemented for file systems of type ffs.

The new API is enabled on a kernel option NEWVNGATE. This option is
not enabled by default in any kernel config.

Presented and discussed on tech-kern with much input from
Bill Studenmund <wrstuden@netbsd.org> and YAMAMOTO Takashi <yamt@netbsd.org>.

Welcome to 4.99.9 (new vfs op vfs_suspendctl).

Consistent usage of KAUTH_GENERIC_ISSUSER.

On snapshot creation be sure the snapshot vnode has valid quota information.

Fixes PR kern/35121

branches: 1.37.2;
ifdef out an unused function if !FFS_NO_SNAPSHOT

__unused removal on arguments; approved by core.

Revisit mnt_vnodelist TAILQ patch. Remove all suspicious TAILQ_FOREACH()
loops where vnodes can get removed or added during the loops. This could
lead to panic's on unmount since nodes are skipped or otherwise
TAILQ_NEXT(0xdeadbeef, ...) was dereferenced.

Replace the LIST structure mp->mnt_vnodelist to a TAILQ structure since all
vnodes were synced and processed backwards. This meant that the last
accessed node was processed first and the earlierst last.

An extra benefit is the removal of the ugly hack from the Berkly days on
LFS.

In the proces, i've also replaced the various variations hand written loops
by the TAILQ_FOREACH() macro's.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

Coverity CID 2949: comment out dead code (from Arnaud Lacombe)

branches: 1.31.4; 1.31.6;
Use the LWP cached credentials where sane.

merge FreeBSD timecounters from branch simonb-timecounters
- struct timeval time is gone
time.tv_sec -> time_second
- struct timeval mono_time is gone
mono_time.tv_sec -> time_uptime
- access to time via
{get,}{micro,nano,bin}time()
get* versions are fast but less precise
- support NTP nanokernel implementation (NTP API 4)
- further reading:
Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf
NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html

branches: 1.29.2;
integrate kauth.

Coverity CID 746: Remove dead code. lbn >= NDADDR is mutually exclusive to
snapshot_locked == 0.

Revert previous; I mixed bpp and *bpp when reading ffs_balloc_ufs1().
ffs_balloc() will always allocate a new buffer or leave it as NULL,
so coverity is wrong here, we're not using a freed argument.

If we brelse ibp, set ibp to NULL, to avoid reusing it later in balloc()
or in our code at the next iteration.
Coverity ID 2706

don't use MALLOC with a non-constant size; use malloc instead.

branches: 1.24.2; 1.24.4; 1.24.6; 1.24.8; 1.24.10;
- add simple functions to allocate/free a buffer for i/o.
- make bufpool static.

branches: 1.23.2;
merge ktrace-lwp.

merge yamt-vop branch. remove following VOPs.

VOP_BLKATOFF
VOP_VALLOC
VOP_BALLOC
VOP_REALLOCBLKS
VOP_VFREE
VOP_TRUNCATE
VOP_UPDATE

branches: 1.21.2;
revert ffs_snapshot.c 1.20 because it's bogus. pointed by Simon Burge.

always use nanotime rather than time.
it's bad to mix nanotime and time because it sometimes
make timestamps go backwards.

64 bit inode changes.

Use ANSI function decls.

branches: 1.17.2;
- sprinkle const
- avoid shadow variables.

- Use an empty snap block list to set the initial file size. Snapshot is
now valid from the beginning. No need to copy the last fs block two times.
- No need to allocate the cylinder group blocks twice.
- cgbuf -> sbbuf

ffs/ffs_alloc.c:
- Add a missing ACTIVECG_CLR().

ffs/ffs_snapshot.c:
- Use async/delayed writes for snapshot creation and sync/uncache these buffers
on end. Reduces the time the file system must be suspended.
- Remove um_snaplistsize. Was a duplicate of um_snapblklist[0].
- Byte swap the list of preallocated blocks on read/write instead of access.
- Always keep this list on ip->i_snapblklist so it may be rolled back when the
newest snapshot gets removed. Fixes a rare snapshot corruption when using
more than one snapshot on a file system.

ufs/ufsmount.h:
- Make TAILQ_LAST() possible on member um_snapshots.
- Remove um_snaplistsize. Was a duplicate of um_snapblklist[0].

Fix last commit. The last block of the file system may have changed
even if the last cylinder group is not modified.

Fix an inconsistency where the last block of the snapshot contains old data.

The last block of the file system is written to the snapshot before the
file system is suspended. If the last cylinder group is modified after
the file system is suspended the last block of the snapshot may contain
old data. So update this block again.

don't assign to non-lvalue. found by gcc4.

branches: 1.11.2;
nuke trailing whitespace

Make `options FFS_NO_SNAPSHOT' only disable snapshot creation
while not trashing existing snapshots.

Approved by: core@

Fss device only checks read access to snapshot vode. On snapshot creation
check we are either super-user or owner of the snapshot vnode.

branches: 1.8.2;
Protect calls to `ffs_*_swap' with `#ifdef FFS_EI'.

branches: 1.7.4;
There's no need to pass a proc value when using UIO_SYSSPACE with
vn_rdwr(9) and uiomove(9).

OK'd by Jason Thorpe

While creating a snapshot inodes must be freed from the
snapshot, not from the file system.
ffs_freefile() needs explicit "fs" and "devvp" arguments.

branches: 1.5.2;
When we expunge an unreferenced file from a snapshot its size may be zero.

- Add flag L_COWINPROGRESS to struct lwp to avoid recursion when
doing copy-on-write.

- Change VFS_SNAPSHOT() to return the snapshot vnode locked.

- Make the IO path for copy-on-write and snapshot-read more lightweight.
Avoids deadlocks where vn_rdwr(...READ...) has a shared lock and needs
to copy-on-write.
Avoids deadlocks/panics where to clean pages the copy-on-write needs
to allocate pages for its VOP_PUTPAGES().

L_COWINPROGRESS part approved by: Jason R. Thorpe <thorpej@netbsd.org>

Once all block address modifications are done invalidate and
free all pages from the snapshot vnode.

Make it compile without option FFS_EI.

Add ffs internal snapshots. Written by Marshall Kirk McKusick for FreeBSD.

- Not enabled by default. Needs kernel option FFS_SNAPSHOT.
- Change parameters of ffs_blkfree.
- Let the copy-on-write functions return an error so spec_strategy
may fail if the copy-on-write fails.
- Change genfs_*lock*() to use vp->v_vnlock instead of &vp->v_lock.
- Add flag B_METAONLY to VOP_BALLOC to return indirect block buffer.
- Add a function ffs_checkfreefile needed for snapshot creation.
- Add special handling of snapshot files:
Snapshots may not be opened for writing and the attributes are read-only.
Use the mtime as the time this snapshot was taken.
Deny mtime updates for snapshot files.
- Add function transferlockers to transfer any waiting processes from
one lock to another.
- Add vfsop VFS_SNAPSHOT to take a snapshot and make it accessible through
a vnode.
- Add snapshot support to ls, fsck_ffs and dump.

Welcome to 2.0F.

Approved by: Jason R. Thorpe <thorpej@netbsd.org>

s/shapshot/snapshot/

Extended attribute support for ffsv2, from FreeBSD.

branches: 1.151.4;
UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.149.10; 1.149.16;
remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.

KASSERT(mutex_owned(vp->v_interlock)) in vnode iterator selector.

#if DIAGNOSTIC panic ---> KASSERT

Remove now redundant calls to fstrans_start()/fstrans_done().

Bring back vrele_flush() to flush deferred vrele() o an suspended file system.

Untangle VFS_SYNC() from VFS_SUSPENDCTL().

branches: 1.143.2;
reorganize ffs_truncate()/ffs_indirtrunc() to be able to partially
succeed; change wapbl_register_deallocation() to return EAGAIN
rather than panic when code hits the limit

callers changed to either loop calling ffs_truncate() using new
utility ufs_truncate_retry() if their semantics requires it, or
just ignore the failure; remove ufs_wapbl_truncate()

this fixes possible user-triggerable panic during truncate, and
resolves WAPBL performance issue with truncates of large files

PR kern/47146 and kern/49175

revert 1.141 - the second ffs_truncate() can't really fail

requested by hannken@

allow also the snapshot_setup()'s call to ffs_truncate() fail, the code
should simply reuse the file blocks in that case; also make sure the
ffs_truncate() call is run within transaction if log is on

branches: 1.140.2;
Small fixes.

ok hannken@

Remove the 'cred' argument from bread(). Remove a now unused var in
ffs_snapshot.c. Update the man page accordingly.

ok hannken@

Remove the 'cred' argument from breadn(), and update the man page
accordingly.

ok hannken@

branches: 1.137.2;
Don't nest structure definitions.

Use an explicit compare to 0 for an immediate error result, not !.
Using ! is perfectly clear on variables like "error" or "result",
but directly on a function call it tends to look like a mistake.

Testing "v_usecount == 1" for exclusive reference will not always
work -- remove and test only readonly.

Introduce a selector function to the vfs vnode iterator so that we don't
need to vget() vnodes that we are not interested at, and optimize locking
a bit. Iterator changes reviewed by Hannken (thanks), the rest of the bugs
are mine.

branches: 1.133.2;
Change snapshot_expunge() to use vfs_vnode_iterator.

ib_get is not used in the evbarm/OPENRD kernel, so mark it as such.

Mark unused (in the !FFS_EI case) variables as such.

Mark a potentially unused (ifndef FFS_EI) variable

Replace macro v_specmountpoint with two functions spec_node_getmountedfs()
and spec_node_setmountedfs() to manage the file system mounted on a device.
Assert the device is a block device.

Welcome to 6.99.24

Discussed on tech-kern@ some time ago.

Reviewed by: David Holland <dholland@netbsd.org>

Kill unused function ib_assign.

branches: 1.127.2;
Stick ffs_ in front of the following macros:
fragstoblks()
blkstofrags()
fragnum()
blknum()

to finish the job of distinguishing them from the lfs versions, which
Christos renamed the other day.

I believe this is the last of the overtly ambiguous exported symbols
from ffs... or at least, the last of the ones that conflicted with lfs.
ffs still pollutes the C namespace very broadly (as does ufs) and this
needs quite a bit more cleanup.

XXX: boo on macros with lowercase names. But I'm not tackling that just yet.

Stick ffs_, ext2_, chfs_, filecore_, cd9660_, or mfs_ in front of
the following symbols so as to disambiguate fully. (Christos already
did the lfs ones.)

lblkno
lblktosize
lfragtosize
numfrags
blkroundup
fragroundup

fsbtodb() -> FFS_FSBTODB(), EXT2_FSBTODB(), or MFS_FSBTODB()
dbtofsb() -> FFS_DBTOFSB() or EXT2_DBTOFSB()

(Christos already did the lfs ones a few days back)

Rename ambiguous macros:
MAXDIRSIZE -> UFS_MAXDIRSIZE or LFS_MAXDIRSIZE
NINDIR -> FFS_NINDIR, EXT2_NINDIR, LFS_NINDIR, or MFS_NINDIR
INOPB -> FFS_INOPB, LFS_INOPB
INOPF -> FFS_INOPF, LFS_INOPF
blksize -> ffs_blksize, ext2_blksize, or lfs_blksize
sblksize -> ffs_blksize

These are not the only ambiguously defined filesystem macros, of
course, there's a pile more. I may not have found all the ambiguous
definitions of blksize(), too, as there are a lot of other things
called 'blksize' in the system.

Add an UFS_SNAPGONE() ufs op replacing the calls
to ffs_snapgone() in ufs_lookup.c.

Ok: David Holland <dholland@netbsd.org>

Welcome to 6.99.22

When invalidating short buffers on the snapshots clean list use bbusy()
to mark the buffer busy. There exists a small window where a buffer is
done but not released and therefore still busy.

Stuff UFS_ in front of a few of ufs's symbols to reduce namespace
pollution. Specifically:
ROOTINO -> UFS_ROOTINO
WINO -> UFS_WINO
NXADDR -> UFS_NXADDR
NDADDR -> UFS_NDADDR
NIADDR -> UFS_NIADDR
MAXSYMLINKLEN -> UFS_MAXSYMLINKLEN
MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Sort out ext2fs's misuse of NDADDR and NIADDR; fortunately, these have
the same values in ext2fs and ffs.

No functional change intended.

Change bread() and breadn() to never return a buffer on
error and modify all callers to not brelse() on error.

Welcome to 6.99.16

PR kern/46282 (6.0_BETA crash: msdosfs_bmap -> pcbmap -> bread -> bio_doread)

branches: 1.119.2;
Replace the remaining KAUTH_GENERIC_ISSUSER authorization calls with
something meaningful. All relevant documentation has been updated or
written.

Most of these changes were brought up in the following messages:

http://mail-index.netbsd.org/tech-kern/2012/01/18/msg012490.html
http://mail-index.netbsd.org/tech-kern/2012/01/19/msg012502.html
http://mail-index.netbsd.org/tech-kern/2012/02/17/msg012728.html

Thanks to christos, manu, njoly, and jmmv for input.

Huge thanks to pgoyette for spinning these changes through some build
cycles and ATF.

branches: 1.118.2; 1.118.6;
As vnalloc() always allocates with PR_WAITOK there is no longer the need
to test its result for NULL.

ffs_copyonwrite(): If the write is to the in-file-system journal
there is no need to lock and check the snapshots.

Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

branches: 1.115.2;
Revert previous commit. Locking the snapshot vnode while the file system
is suspended extends the suspension until the vnode gets unlocked by
the caller of ffs_snapshot().

Resuming the file system before expunging all snapshots and syncing the
snapshot creates races and deadlocks with journaling file systems at least.

Before expunging all snapshots take the snapshot lock and resume the file
system as this is sufficient for the remaining operations.

Reduces the time the file system is suspended and should make this time
independent of the number of snapshots already present.

ffs_snapshot(): return an error if the node is an invalid snapshot.

Preallocate all cylinder group blocks so we no longer redo ~50% of
the cylinder groups while the file system is suspended.
This was removed in error with Rev 1.16.

From Manuel Bouyer <bouyer@netbsd.org> via tech-kern.

merge the bouyer-quota2 branch. This adds a new on-disk format
to store disk quota usage and limits, integrated with ffs
metadata. Usage is checked by fsck_ffs (no more quotacheck)
and is covered by the WAPBL journal. Enabled with kernel
option QUOTA2 (added where QUOTA was enabled in kernel config files),
turned on with tunefs(8) on a per-filesystem
basis. mount_mfs(8) can also turn quotas on.

See http://mail-index.netbsd.org/tech-kern/2011/02/19/msg010025.html
for details.

fss(4): Allow FSSIOCSET to set the initial flags. Add a new flag
"FSS_UNLINK_ON_CREATE" to unlink the backing store before
the snapshot gets created.

With this change dump(8) no longer dumps the zero-sized, but named
snapshot it is working on. Same applies to fsck_ffs(8).

Initialize blkno to 0 right before the snapblkaddr() call that GCC does
not understand so that if ffs_copyonwrite() sprouts a new code path that
does not initialize blkno, the compiler has the chance to reveal it.

Quiesce CC ('blkno' may be used uninitialized in this function).

Move blocks_in_journal() in under #ifndef FFS_NO_SNAPSHOT, all uses
are under that ifdef anyway; this allows build with FFS_NO_SNAPSHOT defined.

Change the snapshot lock:
- No need to take the snapshot lock while the file system is suspended.
- Allow ffs_copyonwrite() one level of recursion with snapshots locked.
- Do the block address lookup with snapshots locked.
- Take the snapshot lock while removing a snapshot from the list.

While hunting deadlocks change the transaction scope for ffs_snapremove().
We could deadlock from UFS_WAPBL_BEGIN() with a buffer held.

Initialize error in snapshot_expunge(); if the list is empty error would
be returned uninitialized. t_snapshot_v2 was failing for me when
librumpffs was compiled DGB=-g.
No idea why gcc didn't catch this ...

Revert rev. 1.101. Dead snapshots would hang around until unmount.

Adresses PR #44568 (WAPBL doens't play nice with snapshots).

Refine the scope of WAPBL transactions so we should no longer get
a "wapbl_flush: current transaction too big to flush" panic when
creating or removing snapshots on larger logging disks.

Adresses PR #44568 (WAPBL doens't play nice with snapshots).

branches: 1.102.2; 1.102.4;
Move counting of faults, traps, intrs, soft[intr]s, syscalls, and nswtch
from uvmexp to per-cpu cpu_data and move them to 64bits. Remove unneeded
includes of <uvm/uvm_extern.h> and/or <uvm/uvm.h>.

Keep a reference to the snapshot vnode until it gets removed from the
snapshot list.

syncsnap: Use bbusy() to take a buffer from v_dirtyblkhd.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

Initialize the initial snap block list's count.

From Antti Kantee <pooka@netbsd.org>.

branches: 1.97.2; 1.97.4;
No longer abuse TAILQ internal data.

Fix a deadlock where fscow_disestablish() blocks because outstanding
copy-on-write operations wait for si_snaplock.

Remove extra whitespace added by a stupid tool.
XXX: more in src/sys/arch

bcopy -> memcpy

bzero -> memset

PR kern/26878 FFSv2 + softdep = livelock (no free ram)
PR kern/16942 panic with softdep and quotas
PR kern/19565 panic: softdep_write_inodeblock: indirect pointer #1 mismatch
PR kern/26274 softdep panic: allocdirect_merge: ...
PR kern/26374 Long delay before non-root users can write to softdep partitions
PR kern/28621 1.6.x "vp != NULL" panic in ffs_softdep.c:4653 while unmounting a softdep (+quota) filesystem
PR kern/29513 FFS+Softdep panic with unfsck-able file-corruption
PR kern/31544 The ffs softdep code appears to fail to write dirty bits to disk
PR kern/31981 stopping scsi disk can cause panic (softdep)
PR kern/32116 kernel panic in softdep (assertion failure)
PR kern/32532 softdep_trackbufs deadlock
PR kern/37191 softdep: locking against myself
PR kern/40474 Kernel panic after remounting raid root with softdep

Retire softdep, pass 2. As discussed and later formally announced on the
mailing lists.

branches: 1.91.2;
merge christos-time_t

Remove superfluous "vp->v_vnlock = &vp->v_lock".

Observed by: YAMAMOTO Takashi <yamt@netbsd.org>

Restore a line removed by mistake with the last commit.

Should fix PR 40225 panic: indiracct: missing indir.

kill MALLOC and FREE macros.

ffs_copyonwrite(): Only use si_snapblklist if it is already allocated.

ffs_snapshot_read(): Use IO_ALTSEMANTICS to allow reading a snapshot vnode
beyond file system size. Needed to read the snapblklist
on mount.

Persistent snapshots work again.

Should fix PR kern/37425: fss_snapshot_mount panic during fsck.

Revert previous -- ALL reads are from kernel space.

Still open: PR kern/37425: fss_snapshot_mount panic during fsck.

ffs_copyonwrite(): Only use si_snapblklist if it is already allocated.
ffs_snapshot_read(): Allow the kernel to read beyond file system size.

Persistent snapshots work again.

Should fix PR kern/37425: fss_snapshot_mount panic during fsck.

Split ffs_freefile into a frontend for normal cylinder group and for
snapshot use. Adjust ffs_blkfree_common to get the fs instance passed
in, the original commit didn't account blocks in the snapshots
correctly. Assert that ffs_blkfree is used with the primary fs instance
and that ffs_checkfreefile is only used for snapshots. Move the bdwrite
from ffs_blkfree_common into the caller for symmetry. This creates a
redundant write of unmodified data for ffs_blkfree_snap if a double free
of a block happens.

Reviewed and tested by hannken@.

ffs_blkfree is used in two different ways. The normal usage is to free a
block in the cylinder groups of the filesystem. The other user is the
snapshot code, which wants to modify the copied cylinder groups. Use
different frontends to distinguish the cases in preparation for fine
grained locking for cylinder groups.

branches: 1.82.2; 1.82.4;
Correct previous.
- Count frags, not blocks to get the file system size.
- Cannot use blksize() here, it depends on vnode size.
- Correctly update xfersize on short reads.

When computing the requests hard limit in ffs_snapshot_read()
use the file system size, not the size of the snapshot vnode.

Adjust some WAPBL transactions:
- Put transaction inside cgaccount() to simplify caller.
- No vget() / vrele() inside a transaction.

Ffs_snapshot() has become a huge monster over the time. Break it into
helper functions to enhance readability. Adjust comments to reality
and test the main error paths.

While here, expand and remove the last FreeBSD->NetBSD conversion macros.

No functional change intended.

Sync the just created snapshot to disk.

Invalidate short ( < fs_bsize ) buffers. We will always read full
size buffers later.

Should fix PR #39402