PR kern/55268: tmpfs is slow

tmpfs_getpages(): handle the PGO_LOCKED case and implement lazy update of
atime/mtime.

s/static inline/static __inline/g for consistency.

branches: 1.54.12;
Change the protocol to update a mounted file system from read-write
to read-only and vice versa:

- Add an internal flag IMNT_WANTRDONLY.
- Set either IMNT_WANTRDWR or IMNT_WANTRDONLY if going from or to read-only.
- After successfull call to VFS_MOUNT() set or clear MNT_RDONLY.

Adapt tmpfs and rumpfs to the new protocol. Other file systems will be
updated when they get the IMNT_CAN_RWTORO property.

Welcome to 7.99.64

Run vflush() when going from read/write to read only.

branches: 1.52.2; 1.52.4;
Change tmpfs to vcache.
- Use tmpfs node address as key.
- Remove tn_vlock, field tn_vnode now protected by vcache.
- Add a hold count to tmpfs node to prevent nodes from disappearing
while tmpfs_fhtovp() trys to vcache_get() them. Last holder
destroys reclaimed nodes.
- Remove the now unneeded parent unlock/lock for lookup of '..'.

Use VFS_PROTOS() for tmpfs.

branches: 1.50.4;
Remove the hardcoded 4 MB free kernel memory limit and replace it
by uvmexp.freetarg, as discussed on tech-kern.
Main purpose is to make tmpfs usable (as far as possible) on small memory
machines.
This is a bit experimental, but we need to give it some real world exposure
to see how well it works.

handle MNT_UPDATE

branches: 1.48.2;
- Simplify tmpfs_update(), eliminate tmpfs_note_t::tn_status and deferred
timestamp updates. Fix some incorrect updates and plug some missing ones.
Should fix PR/48385.
- tmpfs_rmdir: avoid O(n) scan when the directory is not empty and whiteout
entries were never added.

Make tmpfs_node_t::tn_gen a 32-bit number, keep it in sync with tmpfs_fid_t.
Also, change tn_status to unsigned while here.

tmpfs: replace the broken tmpfs_dircookie() logic which uses the node
address truncated to 31 bits (required for 32-bit readdir compatibility,
e.g. linux32). Instead, assign 2^31 range using the following logic:
- The first half of the 2^31 is assigned incrementally (the fast path).
- When exceeded, use the second half of 2^31, but manage with vmem(9).

It will require 2 billion files per-directory to trigger vmem(9) usage.
Also, while here, add some fixes for tmpfs_unmount().

Should fix PR/47739, PR/47480, PR/46088 and PR/41068.
Thanks to wiz@ for stress testing.

branches: 1.45.2; 1.45.12; 1.45.16;
define and use TMPFS_MAXNAMLEN instead of MAXNAMLEN

- Rework and document inode reference counting. Also document inode life
cycle (destruction part). Perform link counting in tmpfs_dir_attach()
and tmpfs_dir_detach(), instead of alloc/free and arbitrary places.
Fixes PR/44285, PR/44288, PR/44657 and likely PR/42484.

- Fix the race between the lookup and inode destruction. Fixes PR/43167
and its duplicates PR/40088, PR/40757.

- Improve tmpfs_rename() locking a little, fix kqueue event notifications
and also fix PR/43617. Add simplistic tmpfs_parentcheck_p(); to be
expanded and used for further rename() locking fixes.

- Cache directory entry "hint" in the tmpfs node, add tmpfs_dir_cached(),
and thus avoid unnecessary lookup in tmpfs_remove() and tmpfs_rmdir().

- Set correct _PC_FILESIZEBITS value in tmpfs_pathconf(). Fixes PR/43576.

- Few minor fixes.

undo the multiple inclusion protection part.

- Prevent tmpfs.h from inclusion in userland.
- Clean up and KNF tmpfs.h a little bit.

- Describe some locking.
- Add VOP argument comments, add some asserts.
- Update/fix/remove outdated/missleading comments.
- Clean up, de-indent, KNF, misc.

No functional changes intended.

- tmpfs: do not create dirent/node pools per-mount, there is no need to.
- tmpfs_mount: fix a leak of mount structures in error path.

branches: 1.39.2;
Add some support for unionfs (not built by default). It's still
missing at least opaque directory support, but until someone figures
out how that should work on ffs (see PR kern/kern/44383), there's
no point in trying to figure out how it should work here.

Replace tmpfs_pool custom allocator code with a simpler layer for memory
accounting. Use wired memory (which can be limited) for meta-data, and
kmem(9) for string allocations.

Close PR/31944. Fix PR/38361 while here. OK ad@.

branches: 1.37.14; 1.37.16;
Solve the fstat-wants-to-look-at-kernel-data-structures in a nicer
way: don't export the fs internals to innocent userspace programs
which just want to mount the file system.

shuffle around some more defs for fstat

noticed by Kurt Schreiner on current-users

Install mount argument structure header just like every other file system.

branches: 1.34.2;
- setting the birthdate is valid in setattr.
- don't call nanotime if not needed.
- don't call tempfs_update() if tempfs_chtimes succeeded, because it just did.

branches: 1.33.2; 1.33.4;
Remove clause 3 and 4 from TNF licenses

branches: 1.32.6; 1.32.8; 1.32.10;
u_int is not a C99 standard type, so spell it out completely as
unsigned int.

Drop trailing whitespace.

Merge vmlocking2 to head.

branches: 1.29.4;
Remove cn_lwp from struct componentname. curlwp should be used
from on. The NDINIT() macro no longer takes the lwp parameter and
associates the credentials of the calling thread with the namei
structure.

branches: 1.28.2;
Back out the tmpfs changes; there appears to be a bug lurking.

Merge from vmlocking.

branches: 1.26.4; 1.26.16; 1.26.18; 1.26.24;
TRUE -> true, FALSE -> false

Replace the Mach-derived boolean_t type with the C99 bool type. A
future commit will replace use of TRUE and FALSE with true and false.

branches: 1.24.4;
Fix typo in comment.

Protect the dircookie stuff with _KERNEL. Fixes build problems of
mount_tmpfs as reported by Hisashi T Fujinaka in private mail.

Truncate directory cookies to 31 bits to avoid problems exposed in Linux
binaries which cast the returned values to 64-bits and fail due to sign
expansion. More details are provided in the big comment in tmpfs.h that
describes how the new tmpfs_dircookie works.

This is a rather ugly hack that shall be fixed with a cleaner solution,
but this resolves the problem in an effective way.

Fixes kern PR/32034.

branches: 1.21.4; 1.21.6;
Use the LWP cached credentials where sane.

branches: 1.20.4;
hide more by ifdef _KERNEL.

branches: 1.19.2;
integrate kauth.

Swap the order of the tf_gen and tf_id members of struct tmpfs_fid,
since tf_id needs 8-byte alignment on some architectures (like amd64).
struct tmpfs_fid now fits in 16 bytes on amd64 again.
OK matt@

Restrict generation numbers to 32bit values - we would have had to bump
_VFS_MAXFIDSZ to 24 otherwise for 64bit archs.

branches: 1.16.2; 1.16.4; 1.16.6;
Change "inline" back to "__inline" in .h files -- C99 is still too
new, and some apps compile things in C89 mode. C89 keywords stay.

As per core@.

Do not use unnamed structures/unions without instances; they break the
build under vax because of gcc 2.95. Found by he@.

make the include files usable from userland.

branches: 1.13.2; 1.13.4; 1.13.6;
Remove a conditional by always using inlined functions in favour of macros.
This way, the code is clearer and there shouldn't be any performance
difference (if the compiler DTRT). Suggested by chs@.

merge ktrace-lwp.

VFS_TO_TMPFS: remove a wrong assertion.
it's wrong because TMPFS_PAGES_MAX can be decreased.

- protect userland exported files against multiple inclusion.
- make sure that kernel only files don't compile in userland using #error
- XXX: some kernel only files still get installed.
- XXX: some files used in userland, don't get installed.

branches: 1.9.2;
merge yamt-vop branch. remove following VOPs.

VOP_BLKATOFF
VOP_VALLOC
VOP_BALLOC
VOP_REALLOCBLKS
VOP_VFREE
VOP_TRUNCATE
VOP_UPDATE

branches: 1.8.2;
Implement the tmpfs_advlock operation. Makes "user-level" file locking
work (aka lockf(3)).

tmpfs_getattr: return up-to-date timestamps.

Kill the tmpfs(9) manual page; it was just documenting internal details of
tmpfs' "API" and was already rotting.

Instead, merge all the relevant comments into the code. This includes
acknowledgements to Google's Summer of Code 2005 program (they were in the
AUTHORS section of tmpfs(9) before), so all the files need to be changed
to include this sentence alongside the title. (Note that this was not a
requirement of the program.)

Apply the NFS exports list rototill patch:

- Remove all NFS related stuff from file system specific code.
- Drop the vfs_checkexp hook and generalize it in the new nfs_check_export
function, thus removing redundancy from all file systems.
- Move all NFS export-related stuff from kern/vfs_subr.c to the new
file sys/nfs/nfs_export.c. The former was becoming large and its code
is always compiled, regardless of the build options. Using the latter,
the code is only compiled in when NFSSERVER is enabled. While doing this,
also make some functions in nfs_subs.c conditional to NFSSERVER.
- Add a new command in nfssvc(2), called NFSSVC_SETEXPORTSLIST, that takes a
path and a set of export entries. At the moment it can only clear the
exports list or append entries, one by one, but it is done in a way that
allows setting the whole set of entries atomically in the future (see the
comment in mountd_set_exports_list or in doc/TODO).
- Change mountd(8) to use the nfssvc(2) system call instead of mount(2) so
that it becomes file system agnostic. In fact, all this whole thing was
done to remove a 'XXX' block from this utility!
- Change the mount*, newfs and fsck* userland utilities to not deal with NFS
exports initialization; done internally by the kernel when initializing
the NFS support for each file system.
- Implement an interface for VFS (called VFS hooks) so that several kernel
subsystems can run arbitrary code upon receipt of specific VFS events.
At the moment, this only provides support for unmount and is used to
destroy NFS exports lists from the file systems being unmounted, though it
has room for extension.

Thanks go to yamt@, chs@, thorpej@, wrstuden@ and others for their comments
and advice in the development of this patch.

change the way to handle directory "offsets" so that
they won't be changed when removing entries in the directory.
some applications like cvs rely on this.

- don't waste/leak kva.
- implement getpages/putpages. support mmap.
- eliminate meaningless memcpy.
- ubcify.

wiz@ remembers that the preferred way to spell file-system in NetBSD is
file system.

Initial addition of tmpfs, an efficient memory file-system. This project
was developed as part of Google's Summer of Code 2005 program. This
change adds the kernel code, the mount_tmpfs utility, a regression test
suite and does all other related changes to integrate these.

The file-system is still *experimental*. Therefore, it is disabled by
default in all kernels. However, as typically done, a commented-out
entry is added in them to ease its setup.

Note that I haven't commited the required mountd(8) changes to be able
to export tmpfs file-systems because NFS support is still very unstable
and because, before enabling it, I'd like to do some other changes.

OK'ed by my project mentor, William Studenmund (wrstuden@).