sys: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

sys: Remove $FreeBSD$: one-line .h pattern

Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/

Fix block bitmap end position computation

PR: 261850
Reported by: chenguang.wang
MFC after: 2 weeks

vfs: add the concept of vnode state transitions

To quote from a comment above vput_final:
<quote>
* XXX Some filesystems pass in an exclusively locked vnode and strongly depend
* on the lock being held all the way until VOP_INACTIVE. This in particular
* happens with UFS which adds half-constructed vnodes to the hash, where they
* can be found by other code.
</quote>

As is there is no mechanism which allows filesystems to denote that a
vnode is fully initialized, consequently problems like the above are
only found the hard way(tm).

Add rudimentary support for state transitions, which in particular allow
to assert the vnode is not legally unlocked until its fate is decided
(either construction finishes or vgone is called to abort it).

The new field lands in a 1-byte hole, thus it does not grow the struct.

Bump __FreeBSD_version to 1400077

Reviewed by: kib (previous version)
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D37759

fs: fix a few common typos in source code comments

- s/quadradically/quadratically/
- s/persistant/persistent/

Obtained from: NetBSD
MFC after: 3 days

ext2fs: minor typo.

Obtained from: Dragonfly
MFC after: 3 days

fs: clean up empty lines in .c and .h files

ext2fs: Stop checking for failures from malloc(M_WAITOK).

PR: 240545
Submitted by: Andrew Reiter <arr@watson.org>
Reviewed by: fsu
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D25707

Add BE architectures support.

Author of most initial version: pfg (https://reviews.freebsd.org/D23259)

Reviewed by: pfg
MFC after: 3 months

Differential Revision: https://reviews.freebsd.org/D24685

Add inode bitmap tail initialization.

Make ext2fs compatible with changes introduced in e2fsprogs v1.45.2.
Now the tail of inode bitmap is filled with 0xff pattern explicitly during
bitmap initialization phase to avoid e2fsck error like:
"Padding at end of inode bitmap is not set."

Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)

r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE. All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by: kib (mentor, blanket)
Commented by: kib, gallatin, melifaro
Differential Revision: https://reviews.freebsd.org/D23718

ext2fs: Remove redundant brelse() after r294954

Coccinelle:

@ rule1 @
identifier __error;
@@
...
int __error;
...

@ rule2 depends on rule1 @
identifier rule1.__error;
identifier __bp;
@@

__error =
(
bread
|
bread_gb
|
breadn
|
breadn_flags
)
(..., &__bp);
if (
(
__error
|
__error != 0
)
) {
...
- brelse(__bp);
...
}

No functional change.

ext2fs: Initial version of DTrace support.

Reviewed by: pfg, gnn
MFC after: 1 week

Differential Revision: https://reviews.freebsd.org/D19848

Remove unneeded mount point unlock function calls.

The ext2_nodealloccg() function unlocks the mount point
in case of successful node allocation.
The additional unlocks are not required and should be removed.

PR: 236452
Reported by: pho
MFC after: 3 days

Do not read the on-disk inode in case of vnode allocation.

Reported by: Christopher Krah <krah@protonmail.com>
Reported as: FS-6-EXT2-4: Denial Of Service in mkdir-0 (ext2_mkdir/vn_rdwr)
Reviewed by: pfg
MFC after: 1 week

Differential Revision: https://reviews.freebsd.org/D19327

Do not panic if inode bitmap is corrupted.

admbug: 804
Reported by: Ilja Van Sprundel <ivansprundel@ioactive.com>
Reviewed by: pfg
MFC after: 1 week

Differential Revision: https://reviews.freebsd.org/D19325

Validate block bitmaps.

Reviewed by: pfg
MFC after: 1 week

Differential Revision: https://reviews.freebsd.org/D19324

Make superblock reading logic more strict.

Add more on-disk superblock consistency checks to ext2_compute_sb_data() function.
It should decrease the probability of mounting filesystems with corrupted superblock data.

Reviewed by: pfg
MFC after: 1 week

Differential Revision: https://reviews.freebsd.org/D19322

Change unused inodes counters behavior in the cylinder groups.
Make it more close to native ext4 implementation to avoid fsck errors.

Fix EXT2FS_DEBUG definition usage.

Reviewed by: pfg
MFC after: 3 months

Differential Revision: https://reviews.freebsd.org/D15394

Revert r313780 (UFS_ prefix)

Prefix UFS symbols with UFS_ to reduce namespace pollution

Followup to r313780. Also prefix ext2's and nandfs's versions with
EXT2_ and NANDFS_.

Reported by: kib
Reviewed by: kib, mckusick
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D9623

Add flex_bg/meta_bg features RW support.

Reviewed by: pfg
MFC after: 6 months

Differential Revision: https://reviews.freebsd.org/D13964

Add metadata_csum feature support.

Reviewed by: pfg (mentor)
Approved by: pfg (mentor)
MFC after: 6 months

Differential Revision: https://reviews.freebsd.org/D13810

Add 64bit feature support.

Reviewed by: kevlo, pfg (mentor)
Approved by: pfg (mentor)
MFC after: 6 months

Differential Revision: https://reviews.freebsd.org/D11530

sys: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Fix physical block number overflow in different places.

Approved by: pfg (mentor)
MFC after: 6 months

Set doreallocblks sysctl value to zero by default because of
possibility of filesystem corruption.

Approved by: pfg (mentor)
MFC after: 2 weeks

Add inital extents read-write support.

Approved by: pfg (mentor)
MFC after: 6 months
RelNotes: Yes

Differential Revision: https://reviews.freebsd.org/D12087

Add check to avoid raw inode iblocks fields overflow in case of huge_file feature.
Use the Linux logic for now.

Reviewed by: pfg (mentor)
Approved by: pfg (mentor)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D12131

ext2fs: Add uninit_bg feature support.

From the linux tune2fs(8) manpage:
"Allow the kernel to initialize bitmaps and inode tables and keep a high
watermark for the unused inodes in a filesystem, to reduce e2fsck(8) time.
This first e2fsck run after enabling this feature will take the full time,
but subsequent e2fsck runs will take only a fraction of the original time,
depending on how full the file system is."

Submitted by: Fedor Uporov
Differential Revision: https://reviews.freebsd.org/D11211

ext2fs: add read-write support for Extended Attributes.

Extended attributes and their particular implementation in linux are
different from FreeBSD so in this case we have started diverging from
the UFS EA implementation, which would be the natural reference.

Depending on future progress implementing ACLs this approach may change
but for now bring to the tree an implementation that is consistent and
can be tested.

Submitted by: Fedor Uporov

Differential Revision: https://reviews.freebsd.org/D10460

prefix UFS symbols with UFS_ to reduce namespace pollution

Specifically:
ROOTINO -> UFS_ROOTINO
WINO -> UFS_WINO
NXADDR -> UFS_NXADDR
NDADDR -> UFS_NDADDR
NIADDR -> UFS_NIADDR
MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Also prefix ext2's and nandfs's NDADDR and NIADDR with EXT2_ and NANDFS_

Reviewed by: kib, mckusick
Obtained from: NetBSD
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D9536

Undo small wrong style change.

Reported by: kib

style(9) cleanups.

Just to reduce some of the issues found with indent(1).

MFC after: 1 week

ext2fs: renumber the license clauses to avoid skipping #3.

This is to keep consistency with other files, and help license-checking
utilities determine the number of clauses that apply.

No functional change.

ext2fs: cleanup generation number management.

Ext2/3/4 manages generation numbers differently than UFS so adopt
some rules that should work well. When allocating a new inode,
make sure we generate a "good" random value specifically avoiding
zero.

Don't interfere with the numbers that are already generated in
the filesystem: ext2fs doesn't have the backwards compatibility
issues where there were no generation numbers.

Reviewed by: kevlo
MFC after: 1 week

arc4random() returns 0 to (2**32)−1, use an alternative to initialize
i_gen if it's zero rather than a divide by 2.

With inputs from delphij, mckusick, rmacklem

Reviewed by: mckusick

ext2fs: replace 0 with NULL for pointers.

While here do late initialization of ebap, similar as was
done in UFS.

Found with devel/coccinelle.

MFC after: 2 weeks

Update comment: Linux does set a randomized generation number of an inode
on ext2/3/4.

While here use arc4random() instead of random().

Reviewed by: pfg
MFC after: 3 days

ext2: Initialize i_flag after allocation.

We use i_flag to carry some flags like IN_E4INDEX which newer
ext2fs variants uses internally.

fsck.ext3 rightfully complains after our implementation tags
non-directory inodes with INDEX_FL.

Initializing i_flag during allocation removes the noise factor
and quiets down fsck.

Patch from: Damjan Jovanovic
PR: 206530

Remove dead code.

After the ext2 variant of the "orlov allocator" was implemented,
the case for a negative or zero dirsize disappeared.

Drop the dead code and unsign dirsize given that it can't be
negative anyways.

CID: 1008669
MFC after: 1 week

Adjust printf format specifiers for dev_t and ino_t in kernel.

ino_t and dev_t are about to become uint64_t.

Reviewed by: kib, mckusick

Revert r263449;

ext2fs: minor update to the dirpref policy.

The change in UFS r254996, reverted the change as the
older code seems to work better. This was not visible
in local testing but we can trust UFS is vastly more
exercised in diferent environments.

ext2fs: minor update to the dirpref policy.

Bring in a minor change to the dirpref policy based on r248623.

This is pretty minimal change to keep the implementation in
sync with UFS but other parts from the original change are not
directly applicable so don't expect improvements in fsck times.

MFC after: 2 weeks

ext2fs: small formatting fixes.

Remove some redundant spaces.
No functional change.

MFC after: 3 days

ext2fs: use of tab vs spaces.

Consistently use a single tab after a #define as mentioned in style(9).
Use tabs instead of space for indenting.
Fix a typo: "hash_vesion".

No functional change.

MFC after: 3 days

ext2fs: Re-enable reallocblk.

The major corruption issues affecting this code have been fixed
a while ago.

MFC after: 1 week

Define ext2fs local types and use them.

Add definitions for e2fs_daddr_t, e4fs_daddr_t in addition
to the already existing e2fs_lbn_t and adjust them for ext4.
Other than making the code more readable these changes should
fix problems related to big filesystems.

Setting the proper types can be tricky so the process was
helped by looking at UFS. In our implementation, logical block
numbers can be negative and the code depends on it. In ext2,
block numbers are unsigned so it is convenient to keep
e2fs_daddr_t unsigned and use the complete 32 bits. In the
case of e4fs_daddr_t, while the value should be unsigned, for
ext4 we only need to support 48 bits so preserving an extra
bit from the sign is not an issue.

While here also drop the ext2_setblock() prototype that was
never used.

Discussed with: mckusick, bde
MFC after: 3 weeks

Define and use e2fs_lbn_t in ext2fs.

In line to what is done in UFS, define an internal type
e2fs_lbn_t for the logical block numbers.

This change is basically a no-op as the new type is unchanged
(int32_t) but it may be useful as bumping this may be required
for ext4fs.

Also, as pointed out by Bruce Evans:

-Use daddr_t for daddr in ext2_bmaparray(). This seems to
improve reliability with the reallocblks option.
- Add a cast to the fsbtodb() macro as in UFS.

Reviewed by: bde
MFC after: 3 days

Re-sort ext2fs headers to make things easier to find.

In the ext2fs driver we have a mixture of headers:

- The ext2_ prefixed headers have strong influence from NetBSD
and are carry specific ext2/3/4 information.
- The unprefixed headers are inspired on UFS and carry implementation
specific information.

Do some small adjustments so that the information is easier to
find coming from either UFS or the NetBSD implementation.

MFC after: 3 days

Relax some unnecessary unsigned type changes in ext2fs.

While the changes in r245820 are in line with the ext2 spec,
the code derived from UFS can use negative values so it is
better to relax some types to keep them as they were, and
somewhat more similar to UFS. While here clean some casts.

Some of the original types are still wrong and will require
more work.

Discussed with: bde
MFC after: 3 days

Turn DIAGNOSTICs to INVARIANTS in ext2fs.

This is done to be consistent with what other filesystems and
particularly ffs already does (see r173464).

MFC after: 5 days

s/file system/filesystem/g

Based on r96755 from UFS.

MFC after: 3 days

Prepare to replace the buf splay with a trie:

- Don't insert BKGRDMARKER bufs into the splay or dirty/clean buf lists.
No consumers need to find them there and it complicates the tree.
These flags are all FFS specific and could be moved out of the buf
cache.
- Use pbgetvp() and pbrelvp() to associate the background and journal
bufs with the vp. Not only is this much cheaper it makes more sense
for these transient bufs.
- Fix the assertions in pbget* and pbrel*. It's not safe to check list
pointers which were never initialized. Use the BX flags instead. We
also check B_PAGING in reassignbuf() so this should cover all cases.

Discussed with: kib, mckusick, attilio
Sponsored by: EMC / Isilon Storage Division

ext2fs: Use prototype declarations for function definitions

Submitted by: Christoph Mallon
MFC after: 2 weeks

ext2fs: general cleanup.

- Remove unused extern declarations in fs.h
- Correct comments in ext2_dir.h
- Several panic() messages showed wrong function names.
- Remove commented out stray line in ext2_alloc.c.
- Remove the unused macro EXT2_BLOCK_SIZE_BITS() and the then
write-only member e2fs_blocksize_bits from struct m_ext2fs.
- Remove the unused macro EXT2_FIRST_INO() and the then write-only
member e2fs_first_inode from struct m_ext2fs.
- Remove EXT2_DESC_PER_BLOCK() and the member e2fs_descpb from
struct m_ext2fs.
- Remove the unused members e2fs_bmask, e2fs_dbpg and
e2fs_mount_opt from struct m_ext2fs
- Correct harmless off-by-one error for fspath in ext2_vfsops.c.
- Remove the unused and broken macros EXT2_ADDR_PER_BLOCK_BITS()
and EXT2_DESC_PER_BLOCK_BITS().
- Remove the !_KERNEL versions of the EXT2_* macros.

Submitted by: Christoph Mallon
MFC after: 2 weeks

ext2fs: make some inode fields match the ext2 spec.

Ext2fs uses unsigned fields in its dinode struct.
FreeBSD can have negative values in some of those
fields and the inode is meant to interact with the
system so we have never respected the unsigned
nature of most of those fields.

Block numbers and the NFS generation number do
not need to be signed so redefine them as
unsigned to better match the on-disk information.

MFC after: 1 week

ext2fs: temporarily disable the reallocation code.

Testing with fsx has revealed problems and in order to
hunt the bugs properly we need reduce the complexity.

This seems to help but is not a complete solution.

MFC after: 3 days

Partially bring r242520 to ext2fs.

When a file is first being written, the dynamic block reallocation
(implemented by ext2_reallocblks) relocates the file's blocks
so as to cluster them together into a contiguous set of blocks on
the disk.

When the cluster crosses the boundary into the first indirect block,
the first indirect block is initially allocated in a position
immediately following the last direct block. Block reallocation
would usually destroy locality by moving the indirect block out of
the way to keep the data blocks contiguous.

The issue was diagnosed long ago by Bruce Evans on ffs and surfaced
on ext2fs when block reallocaton was ported. This is only a partial
solution based on the similarities with FFS. We still require more
review of the allocation details that vary in ext2fs.

Reported by: bde
MFC after: 1 week

Fix up kernel sources to be ready for a 64-bit ino_t.

Original code by: Gleb Kurtsou

Add support for ns timestamps and birthtime to the ext2/3 driver.

When using big inodes there is sufficient space in ext3 to
keep extra resolution and birthtime (creation) timestamps.
The appropriate fields in the on-disk inode have been approved
for a long time but support for this in ext3 has not been
widely distributed.

In preparation for ext4 most linux distributions have enabled
by default such bigger inodes and some people use nanosecond
timestamps in ext3. We now support those when the inode is big
enough and while we do recognize the EXT4F_ROCOMPAT_EXTRA_ISIZE,
we maintain the extra timestamps even when they are not used.

An additional note by Bruce Evans:
We blindly accept unrepresentable tv_nsec in VOP_SETATTR(), but
all file systems have always done that. When POSIX gets around
to specifying the behaviour, it will probably require certain
rounding to the fs's resolution and not rejecting the request.
This unfortunately means that syscalls that set times can't
really tell if they succeeded without reading back the times
using stat() or similar and checking that they were set close
enough.

Reviewed by: bde
Approved by: jhb (mentor)
MFC after: 2 weeks

Migrate ufs and ext2fs from skpc() to memcchr().

While there, remove a useless check from the code. memcchr() always
returns characters unequal to 0xff in this case, so inosused[i] ^ 0xff
can never be equal to zero. Also, the fact that memcchr() returns a
pointer instead of the number of bytes until the end, makes conversion
to an offset far more easy.

Style cleanups by jh@.
Fix a comment from the previous commit.
Use M_ZERO instead of bzero() in ext2_vfsops.c
Add include guards from PR.

PR: 162564
Approved by: jhb (mentor)
MFC after: 2 weeks

Bring in reallocblk to ext2fs.

The feature has been standard for a while in UFS as a means to reduce
fragmentation, therefore maintaining consistent performance with
filesystem aging. This is also very similar to what ext4 calls
"delayed allocation".

In his 2010 GSoC, Zheng Liu ported and benchmarked the missing
FANCY_REALLOC code to find more consistent performance improvements than
with the preallocation approach.

PR: 159233
Author: Zheng Liu <gnehzuil AT SPAMFREE gmail DOT com>
Sponsored by: Google Inc.
Approved by: jhb (mentor)
MFC after: 2 weeks

Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.

The SYSCTL_NODE macro defines a list that stores all child-elements of
that node. If there's no SYSCTL_DECL macro anywhere else, there's no
reason why it shouldn't be static.

Various style fixes including using uint*_t instead of u_int*_t.

Submitted by: Pedro F. Giffuni giffunip at yahoo

Use ffs() to locate free bits in the inode and block bitmaps rather than
loops with bit shifts.

After reading a bitmap block for i-nodes or blocks, recheck the count of
free i-nodes or blocks to handle a race where another thread might have
allocated the last i-node or block while we were waiting for the buffer.

Tested by: dougb

Fix build with DIAGNOSTIC enabled.

Pointy hat to: jhb

- Set the next_alloc fields for an i-node after allocating a new block
so that future allocations start with most recently allocated block
rather than the beginning of the filesystem.
- Fix ext2_alloccg() to properly scan for 8 block chunks that are not
aligned on 8-bit boundaries. Previously this was causing new blocks
to be allocated in a highly fragmented fashion (block 0 of a file at
lbn N, block 1 at lbn N + 8, block 2 at lbn N + 16, etc.).
- Cosmetic tweaks to the currently-disabled fancy realloc sysctls.

PR: kern/153584
Discussed with: bde
Tested by: Pedro F. Giffuni giffunip at yahoo, Zheng Liu (lz)

Whitespace and style fixes.

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

- Remove duplicate comment.

PR: kern/148820
Submitted by: pluknet <pluknet - at - gmail.com>

Bring in the ext2fs work done by Aditya Sarawgi during and after Google Summer
of Code 2009:

- BSDL block and inode allocation policies for ext2fs. This involves the use
FFS1 style block and inode allocation for ext2fs. Preallocation was removed
since it was GPL'd.
- Make ext2fs MPSAFE by introducing locks to per-mount datastructures.
- Fixes for kern/122047 PR.
- Various small bugfixes.
- Move out of gnu/ directory.

Sponsored by: Google Inc.
Submitted by: Aditya Sarawgi <sarawgi.aditya AT SPAMFREE gmail DOT com>