Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Merge checked by: Glen Barber (gjb@)

MFC of 247212:
When running with the -d option, instrument fsck_ffs to track the number,
data type, and running time of its I/O operations.

No functional changes.

MFC of 247234:
Catch up with internal API changes for initbarea() and getdatablk()
of fsck_ffs introduced with r247212.

Submitted by: David Wolfskill <david@catwhisker.org>

MFC of 248625:
Speed up fsck by caching the cylinder group maps in pass1 so
that they do not need to be read again in pass5. As this nearly
doubles the memory requirement for fsck, the cache is thrown away
if other memory needs in fsck would otherwise fail. Thus, the
memory footprint of fsck remains unchanged in memory constrained
environments.

This work was inspired by a paper presented at Usenix's FAST '13:
www.usenix.org/conference/fast13/ffsck-fast-file-system-checker

Details of this implementation appears in the April 2013 of ;login:
www.usenix.org/publications/login/april-2013-volume-38-number-2.
A copy of the April 2013 ;login: paper can also be downloaded
from: www.mckusick.com/publications/faster_fsck.pdf.

Reviewed by: kib
Tested by: Peter Holm

MFC of 248639:
Fix the build after addition of cylinder group cacheing (r248625)

Reported by: Glen Barber (gjb@)
Pointy hat to: Kirk McKusick (mckusick@)

MFC of 248673:
Minor formatting fix for printf() to fix clang builds.

Submitted by: db
Reviewed by: gjb

MFC of 248680:
Resolve clang compile errors on amd64/i386 for certain by casting.

compile tested with clang on i386, amd64
compile tested with gcc on i386, amd64, sparc64

Submitted by: delphij

MFC of 248691:
Note that output is in seconds, not msec.
KNF indentation.
No functional change.
No change to printf strings.
No change to casting of printf arguments.

Reported by: Bruce Evans

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Mechanical whitespace cleanup.

MFC after: 3 weeks

The dump, fsck_ffs, fsdb, fsirand, newfs, makefs, and quot utilities
include sys/time.h instead of time.h. This include is incorrect as
per the manpages for the APIs and the POSIX definitions. This commit
replaces sys/time.h where necessary with time.h.

The commit also includes some minor style(9) header fixup in newfs.

This commit is part of a larger effort by Garrett Cooper started in
//depot/user/gcooper/posix-conformance-work/ -- to make FreeBSD more
POSIX compliant.

Submitted by: Garrett Cooper yanegomi at gmail dot com

Update the actions previously attempted by the -D option to make them
robust. With these changes fsck is now able to detect and reliably
rebuild corrupted cylinder group maps. The -D option is no longer
necessary as it has been replaced by a prompt asking whether the
corrupted cylinder group should be rebuilt and doing so when requested.
These actions are only offered and taken when running fsck in manual
mode. Corrupted cylinder groups found during preen mode cause the fsck
to fail.

Add the -r option to free up excess unused inodes. Decreasing the
number of preallocated inodes reduces the running time of future
runs of fsck and frees up space that can allocated to files. The -r
option is ignored when running in preen mode.

Reviewed by: Xin LI <delphij@>
Sponsored by: Rsync.net

Add a new flag, '-C' which enables a special mode that is intended for
catastrophic recovery. Currently, this mode only validates whether a
cylindergroup has good signature data, and prompts the user to decide
whether to clear it as a whole.

This mode is useful when there is data damage on a disk and you are
working on copy of the original disk, as fsck_ffs(8) tends to abnormally
exit in such case, as a last resort to recover data from the disk.

Implements gjournal support. If file system has gjournal support enabled
and -p flag was given perform fast file system checking (bascially only
garbage collecting of orphaned objects).

Rename bread() to blread() and bwrite() to blwrite() as we now link to
the libufs library, which also implement functions with that names.

Sponsored by: home.pl

Eliminate linked list used to track inodes with an initial link
count of zero and instead encode this information in the inode state.
Pass 4 performed a linear search of this list for each inode in
the file system, which performs poorly if the list is long.

Reviewed by: sam & keramida (an earlier version of the patch), mckusick
MFC after: 1 month

Create DIP_SET() and IBLK_SET() macros to fix lvalue warnings.

Inspired by: kan

Remove advertising clause from University of California Regent's license,
per letter dated July 22, 1999.

Approved by: core, imp

Use __FBSDID() to quiet GCC 3.3 warnings.

Fix a bunch of format string warnings which broke
the sparc64 build.

Tested on: sparc64, i386

Warning cleanup.

Format changes by peter

This commit adds basic support for the UFS2 filesystem. The UFS2
filesystem expands the inode to 256 bytes to make space for 64-bit
block pointers. It also adds a file-creation time field, an ability
to use jumbo blocks per inode to allow extent like pointer density,
and space for extended attributes (up to twice the filesystem block
size worth of attributes, e.g., on a 16K filesystem, there is space
for 32K of attributes). UFS2 fully supports and runs existing UFS1
filesystems. New filesystems built using newfs can be built in either
UFS1 or UFS2 format using the -O option. In this commit UFS1 is
the default format, so if you want to build UFS2 format filesystems,
you must specify -O 2. This default will be changed to UFS2 when
UFS2 proves itself to be stable. In this commit the boot code for
reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c)
as there is insufficient space in the boot block. Once the size of the
boot block is increased, this code can be defined.

Things to note: the definition of SBSIZE has changed to SBLOCKSIZE.
The header file <ufs/ufs/dinode.h> must be included before
<ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and
ufs_lbn_t.

Still TODO:
Verify that the first level bootstraps work for all the architectures.
Convert the utility ffsinfo to understand UFS2 and test growfs.
Add support for the extended attribute storage. Update soft updates
to ensure integrity of extended attribute storage. Switch the
current extended attribute interfaces to use the extended attribute
storage. Add the extent like functionality (framework is there,
but is currently never used).

Sponsored by: DARPA & NAI Labs.
Reviewed by: Poul-Henning Kamp <phk@freebsd.org>

UFS2 preparation commit:
Remove support for converting old FFS formats to newer.

Submitted by: mckusick
Sponspored by: DARPA & NAI Labs.

o __P removed
o ansi function prototypes
o unifdef -D__STDC__
o __dead2 on usage prototype
o remove now-bogus main prototype

Remove 'register' keyword.
It does not help modern compilers, and some may take some hit from it.
(I also found several functions that listed *every* of its 10 local vars with
"register" -- just how many free registers do people think machines have?)

declare locally used globals as static.

Fix a large number of -Wall, -Wformat and -W compiler warnings.
These were mainly missing casts or wrong format strings in printf
statements, but there were also missing includes, unused variables,
functions and arguments.

The choice of `long' vs `int' still seems almost random in a lot
of places though.

Silence non-constant format string warnings by marking functions
as __printflike()/__printf0like(), adding const, or adding missing "%s"
format strings, as appropriate.

MFC after: 2 weeks

Just notify us once when encountering a partially allocated inode.

Additions to run checks on live filesystems. This change will not
affect current systems until fsck is modified to use these new
facilities. To try out this change, set the fsck passno to zero
in /etc/fstab to cause the filesystem to be mounted without running
fsck, then run `fsck_ffs -p -B <filesystem>' after the system has
been brought up multiuser to run a background cleanup on <filesystem>.
Note that the <filesystem> in question must have soft updates enabled.

Make a tighter test for valid inode numbers in getnextinode().

Teach fsck about snapshot files. These changes should have no
effect on operation of fsck on filesystems without snapshots.
If you get compilation errors, be sure that you have copies of
/usr/include/sys/mount.h (1.94), /usr/include/sys/stat.h (1.21),
and /usr/include/ufs/ffs/fs.h (1.16) as of July 4, 2000 or later.

Yesterday I had to fix a badly broken disk, and found that fsck kept dying:

DIR I=64512 CONNECTED. PARENT WAS I=4032
fsck: cannot find inode 995904

fsdb found the inodes with no problem:

fsdb (inum: 64512)> inode 995904
current inode: directory
I=995904 MODE=40777 SIZE=512
MTIME=Feb 14 15:27:07 2000 [0 nsec]
CTIME=Feb 14 15:27:07 2000 [0 nsec]
ATIME=Feb 24 10:31:58 2000 [0 nsec]
OWNER=nobody GRP=nobody LINKCNT=4 FLAGS=0 BLKCNT=2 GEN=38a41386
Direct blocks: 8094568 0 0 0 0 0 0 0 0 0 0 0
Indirect blocks: 0 0 0

The problem turns out to be a program logic error in fsck. It stores
directory inodes internally in hash lists, using the number of
directories to form the hash key:

inpp = &inphead[inumber % numdirs];

Elsewhere, however, it increments numdirs when it finds unattached
directories. I've made the following fix, which solved the problem in
the case in hand.

Submitted by: Greg Lehey <grog@lemis.com>
Reviewed by: Matthew Dillon <dillon@apollo.backplane.com>
Approved by: Kirk McKusick <mckusick@mckusick.com>

$Id$ -> $FreeBSD$

Cosmetic and documentation changes brought from earlier FreeBSD versions.
(e.g. RCS Id:)

Reviewed by: Don Lewis <Don.Lewis@tsc.tdk.com>
Submitted by: Kirk McKusick <mckusick@McKusick.COM>
Obtained from: Mckusick, BSDI and a host of others

This exactly matches Kirks sources imported under the
Tag MCKUSICK2. These are as supplied by kirk with one small
change needed to compile under freeBSD.

Some FreeBSD patches will be added back, though many have been
added to Kirk's sources already.

Avoid trying to malloc > (1<<32) bytes of memory due to an arithmetic
underflow on the alpha.

Fixed printf format errors.

Correct use of .Nm. Add rcsid. Remove unused #includes. Use err(3).

Reviewed by: dyson@freebsd.org (john Dyson), dg@root.com (david greenman)
Submitted by: Kirk McKusick (mcKusick@mckusick.com)
Obtained from: WHistle development tree

Fixed overflow in chkrange(). Some out of bounds block numbers,
e.g. -1, were not detected. Use a bulletproof check that doesn't
depend on special properties of the args or the limit.

PR: 3528

patch up some "int *" vs. "time_t *" (long) mismatches. They could be
nasty if sizeof(int) != sizeof(long).

Finished (?) merging with Lite2: cleaned up #include mess and fixed a
style bug.

Removed a redundant declaration.

Merge from Lite2. Note that Lite2 has it's own filesystem clean check
skipping code that overrides ours sooner. One should be eliminated,
but for now it works.

Fix the case where fsck would not see sparse directories and the kernel would
panic. If such a thing is fixed fsck needs a rerun (and bugs the user to do
so).

Reviewed by: Kirk McKusick

ts_sec -> tv_sec
ts_nsec -> tv_nsec

Remove trailing whitespace.

Submitted by: Philippe Charnier <charnier@lirmm.fr>, distilled by bde

Fix a couple more bogus types that aren't reported by `gcc -Wall'.

Submitted by: phk, added to by bde

Fix all the warnings from `gcc -Wall'.

>The fix for the missing ".." in the root directory is enclosed below.

Submitted by: Kirk McKusick

Fixed so that it can grok old style "fastlinks".

BSD 4.4 Lite sbin Sources

Note: XNSrouted and routed NOT imported here, they shall be imported with
usr.sbin.