MFC -fno-common fixes: r359389, r359394, r359397-r359399, r359403-r359404,
r359406, r359413-r359416, r359425, r359427, r359432-r359433, r359443,
r359675-r359677

Note: this is not necessarily a complete fix to get these programs to build
with -fno-common applied.

r359389:
config(8): fixes for -fno-common

Move this handful of definitions into main.c, properly declare these as
extern in config.h. This fixes the config(8) build with -fno-common.

Unexplained in my previous commit to gas, -fno-common will become the
default in GCC10 and LLVM11, so it's worth addressing these in advance.

r359394:
MFV r359393: tcsh: import 6974bc35a5cd

This removes an extra variable definition that causes the -fno-common build
to fail, which will be a new default in GCC10/LLVM11.

r359397:
zfs: fix -fno-common issues

A similar (or identical?) fix has already landed in OpenZFS.

-fno-common will become the default in GCC10/LLVM11.

r359398:
sh: remove duplicate el definition

el is declared extern in myhistedit.h and defined in histedit.c. Remove the
duplicate definition in input.c to appease the -fno-common build.

-fno-common will become the default in GCC10/LLVM11.

r359399:
telnet: remove some duplicate definitions, mark terminaltype extern

Most of these were already properly declared and defined elsewhere, this is
effectively just a minor cleanup that fixes the -fno-common build.

-fno-common will become the default in GCC10/LLVM11.

r359403:
Revert 359399: telnet -fno-common bits

There was a large misfire from my local diff that I need to investigate, and
this version committed did not build.

r359404:
Re-apply r359399: telnet -fno-common fix

line and auth_level's redefinitions are just extraneous

telnetd will #define extern and then include ext.h to allocate storage for
all of these extern'd vars; however, two of them are actually defined in
libtelnet instead. Instead of doing an #ifdef extern dance around those
function pointers, just add an EXTERN macro to make it easier to
differentiate by sight which ones will get allocated in globals.c and which
ones are defined elsewhere.

r359406:
telnet: kill off remaining duplicate definition

r359413:
ipfilter: remove duplicate definition of 'thishost'

thishost is already defined in lib/initparse.c; no need for this one. This
fixes the ipfilter build with -fno-common.

-fno-common will become the default in GCC10/LLVM11.

r359414:
iscontrol: move definition of vflag/iscsidev to iscontrol.c

Mark the declaration extern as these are used elsewhere; this fixes the
build with -fno-common.

r359415:
userboot: mark host_fsops as extern

This is already defined elsewhere; mark this declaration extern to the fix
the -fno-common build.

r359416:
systat: remove redundant definition of kd

kd is already properly declared in extern.h and defined in main.c, rendering
this definition useless. This fixes the -fno-common build.

r359425:
locate: fix -fno-common build

Just a single variable declaration to extern and define elsewhere here,
myctype.

-fno-common will become a default in GCC10/LLVM11.

r359427:
fsck_ffs/fsdb: fix -fno-common build

This one is also a small list:

- 3x duplicate definition (ufs2_zino, returntosingle, nflag)
- 5x 'needs extern', 3/5 of which are referenced in fsdb

-fno-common will become the default in GCC10/LLVM11.

r359432:
gdb: compile with -fcommon explicitly

As described in the comment, gdb relies on some of the linker magic that
happens with -fcommon. I suspect the life expectancy of gdb-in-base is low
enough that this isn't worth spending much time addressing, especially given
the vintage. Hit it with the -fcommon hammer so that it continues to just
work.

r359433:
bmake: fix -fno-common build

debug was declared extern, but debug_file was not; correct this and define
debug_file in main.c (as debug is) to fix the -fno-common build.

-fno-common will become the default with GCC10/LLVM11.

r359443:
MFV r359442: bmake: import -fno-common fix build back from upstream

sjg@ committed the local patch previously committed upstream; pull it in to
vendor/ to ease any potential stress of future imports.

r359675:
kqueue tests: fix -fno-common build

vnode_fd and kqfd are both shared among multiple CU; define them exactly
once.

In the case of vnode_fd, it was simply the declaration that needed
correction.

-fno-common will become the default in GCC10/LLVM11.

r359676:
ntpd: fix build with -fno-common

Only a small nit here: psl should be declared extern and defined exactly
once.

-fno-common will become the default in GCC10/LLVM11.

r359677:
yp*: fix -fno-common build

This is mostly two problems spread out far and wide:
- ypldap_process should be declared properly
- debug is defined differently in many programs

For the latter, just extern it and define it everywhere that actually needs
it. This mostly works out nicely for ^/libexec/ypxfr, which can remove the
assignment at the beginning of main in favor of defining it properly.

-fno-common will become the default in GCC10/LLVM11.

MFC r323157 by 323157: fix recovery information with sector sizes up to 64K.

Original commit log:

The new fsck recovery information to enable it to find backup
superblocks created in revision 322297 only works on disks
with sector sizes up to 4K. This update allows the recovery
information to be created by newfs and used by fsck on disks
with sector sizes up to 64K. Note that FFS currently limits
filesystem to be mounted from disks with up to 8K sectors.
Expanding this limitation will be the subject of another
commit.

For example, this allows newfs to work on GELI volumes with 8K sectors.

PR: 243413
Approved by: mckusick
Relnotes: Yes

MFC of 344552 and 344732

Have fsck_ffs adjust size for files with hole at end

Tighten last lbn calculation

Sponsored by: Netflix

Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

MFC of 322200, 322201, 322271, and 322297

322200: Remove (broken) search for alternate superblocks
322201: Show differences when alternate superblock fails to match
322271: Cleanup for 322200.
322297: Restore fsck_ffs ability to find alternate superblocks

Discussed with: kib, imp
Differential Revision: https://reviews.freebsd.org/D11589

MFC r317283:
fsck_ffs: Unsign some variables and make use of reallocarray(3).

Instead of casting listmax and numdirs to unsigned values just define
them as unsigned and avoid the casts. Use reallocarray(3).

While here, fs_ncg is already unsigned so the cast is unnecessary.

Reviewed by: mckusick

MFC 304438:

Fsck_ufs was using an int rather than a ufs2_daddr_t to store the
alternate superblock location when given in the -b option.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Use MIN/MAX macros from sys/param.h.

MFC after: 2 weeks.

Remove old ioctl use and support, once and for all.

Revert 248634 and 248643 (e.g., restoring 248625 and 248639).

Build verified by: Glen Barber (gjb@)

Revert svn r248625

Clang errors around printf could be trivially fixed, but the breakage in
sbin/fsdb were to significant for this type of change.

Submitter of this changeset has been notified and hopefully this can be
restored soon.

Speed up fsck by caching the cylinder group maps in pass1 so
that they do not need to be read again in pass5. As this nearly
doubles the memory requirement for fsck, the cache is thrown away
if other memory needs in fsck would otherwise fail. Thus, the
memory footprint of fsck remains unchanged in memory constrained
environments.

This work was inspired by a paper presented at Usenix's FAST '13:
www.usenix.org/conference/fast13/ffsck-fast-file-system-checker

Details of this implementation appears in the April 2013 of ;login:
www.usenix.org/publications/login/april-2013-volume-38-number-2.
A copy of the April 2013 ;login: paper can also be downloaded
from: www.mckusick.com/publications/faster_fsck.pdf.

Reviewed by: kib
Tested by: Peter Holm
MFC after: 4 weeks

When running with the -d option, instrument fsck_ffs to track the number,
data type, and running time of its I/O operations.

No functional changes.

Be more helpful about alternate superblocks.

In checker, read journal by sectors.

Due to UFS insistence to pretend that device sector size is 512 bytes,
sector size is obtained from ioctl(DIOCGSECTORSIZE) for real devices,
and from the label otherwise. The file images without label have to
be made with 512 sector size.

In collaboration with: pho
Reviewed by: jeff
Tested by: bz, pho

Add the '-C' "check clean" flag. If the FS is marked clean, skip file
system checking. However, if the file system is not clean, perform a
full fsck.

Reviewed by: delphij
Obtained from: Juniper Networks

Be more careful when checking superblock. We have already checked
whether fs_bsize is larger than MINBSIZE, which is larger than the
value that is used to compared with fs_bsize, the sizeof fs, so the
check followed, will be always true.

By inspecting the code and some old commit log, I believe that the
check must be that *fs_sbsize* is larger than sizeof fs. We round
up the size to nearest dev_bsize, as the smallest accepted fs_sbsize,
personally, I think this can be even changed to equal, because this
number is mostly an invariant in file systems.

With this check, fsck_ffs(8) will be more picky and has better
chance rejecting bad first superblock rather than referring to bad
value it supplied, thus gives better chance for it to check the
filesystem carefully.

Implements gjournal support. If file system has gjournal support enabled
and -p flag was given perform fast file system checking (bascially only
garbage collecting of orphaned objects).

Rename bread() to blread() and bwrite() to blwrite() as we now link to
the libufs library, which also implement functions with that names.

Sponsored by: home.pl

Make background fsck based summary adjustments actually work by
initializing the sysctl mibs data before actually using them.

The original patchset (which is the actual version that is running
on my testboxes) have checked whether all of these sysctls and
refuses to do background fsck if we don't have them. Kirk has
pointed out that refusing running fsck on old kernels is pointless,
as old kernels will recompute the summary at mount time, so I
have removed these checks.

Unfortunatelly, as the checks will initialize the mib values of
those sysctl's, and which are vital for the runtime summary
adjustment to work, we can not simply remove the check, which
will lead to problem when running background fsck over a dirty
volume. Add these checks in a different way: give a warning rather
than refusing to work, and complain if the functionality is not
available when adjustments are necessary.

Noticed by: A power failure at my lab
Pointy hat: me
MFC After: 3 days

Generalize the UFS bad magic value used to determine when a filesystem
has only been partly initialized via newfs(8) so that it applies to both
UFS1 and UFS2.

Submitted by: "Xin LI" delphij at frontfree dot net
MFC: maybe?

Remove advertising clause from University of California Regent's license,
per letter dated July 22, 1999.

Approved by: core, imp

Avoid dereferencing null pointers in fsck_ffs. (pfatal may return,
so it isn't a safe way of handling [mc]alloc failures.)

PR: misc/61800
Approved by: rwatson (mentor)

Catch and report on filesystems that were interrupted during newfs,
sporting the new 'BAD' magic number. Exit with a unique error code
(11) so callers who care about this can respond appropriately.

Spell "file system" correctly.

Use __FBSDID() to quiet GCC 3.3 warnings.

Verify that alternate superblocks have a correct magic number before
trying to use them. Set a minimum value for numdirs when using an
alternate superblock to avoid spurious numdirs == 0 error. Calculate
new fields when using an alternate superblock from a UFS1 filesystem
to avoid segment faulting.

Sponsored by: DARPA & NAI Labs.

Create a new 32-bit fs_flags word in the superblock. Add code to move
the old 8-bit fs_old_flags to the new location the first time that the
filesystem is mounted by a new kernel. One of the unused flags in
fs_old_flags is used to indicate that the flags have been moved.
Leave the fs_old_flags word intact so that it will work properly if
used on an old kernel.

Change the fs_sblockloc superblock location field to be in units
of bytes instead of in units of filesystem fragments. The old units
did not work properly when the fragment size exceeeded the superblock
size (8192). Update old fs_sblockloc values at the same time that
the flags are moved.

Suggested by: BOUWSMA Barry <freebsd-misuser@netscum.dyndns.dk>
Sponsored by: DARPA & NAI Labs.

When verifying a superblock, the lower bound on block size is MINBSIZE
(4096), not SBLOCKSIZE (8192).

Submitted by: Tor.Egge@cvsup.no.freebsd.org
Sponsored by: DARPA & NAI Labs.

Use the standardized CHAR_BIT constant instead of NBBY in userland.

s/filesystem/file system/g as discussed on -developers

Fix more breakage due to FSTYPENAMES/DKTYPENAMES split.

Also submitted by: David Wolfskill <david@catwhisker.org>

Don't '#define DKTYPENAMES', we don't use the variable it exposes from
the header file.

Fix a bunch of format string warnings which broke
the sparc64 build.

Tested on: sparc64, i386

Warning cleanup.

Format changes by peter

This commit adds basic support for the UFS2 filesystem. The UFS2
filesystem expands the inode to 256 bytes to make space for 64-bit
block pointers. It also adds a file-creation time field, an ability
to use jumbo blocks per inode to allow extent like pointer density,
and space for extended attributes (up to twice the filesystem block
size worth of attributes, e.g., on a 16K filesystem, there is space
for 32K of attributes). UFS2 fully supports and runs existing UFS1
filesystems. New filesystems built using newfs can be built in either
UFS1 or UFS2 format using the -O option. In this commit UFS1 is
the default format, so if you want to build UFS2 format filesystems,
you must specify -O 2. This default will be changed to UFS2 when
UFS2 proves itself to be stable. In this commit the boot code for
reading UFS2 filesystems is not compiled (see /sys/boot/common/ufsread.c)
as there is insufficient space in the boot block. Once the size of the
boot block is increased, this code can be defined.

Things to note: the definition of SBSIZE has changed to SBLOCKSIZE.
The header file <ufs/ufs/dinode.h> must be included before
<ufs/ffs/fs.h> so as to get the definitions of ufs2_daddr_t and
ufs_lbn_t.

Still TODO:
Verify that the first level bootstraps work for all the architectures.
Convert the utility ffsinfo to understand UFS2 and test growfs.
Add support for the extended attribute storage. Update soft updates
to ensure integrity of extended attribute storage. Switch the
current extended attribute interfaces to use the extended attribute
storage. Add the extent like functionality (framework is there,
but is currently never used).

Sponsored by: DARPA & NAI Labs.
Reviewed by: Poul-Henning Kamp <phk@freebsd.org>

UFS2 preparation commit:
Remove support for converting old FFS formats to newer.

Submitted by: mckusick
Sponspored by: DARPA & NAI Labs.

Retire the bogus uses of the disklabel field d_sbsize and begin to
initialize it to zero so we don't have to have everbody and their
aunt including FFS specific header files.

Sponsored by: DARPA & NAI Labs.

When checking the alternate superblock, we used to copy any fields
that might have changed, then did a byte-by-byte comparison with
the alternate. If any unused fields got used, they had to be added
to the exception list. Such changes caused too many false alarms.
So, I have changed the comparison algorithm to compare a selected
set of fields that are not expected to change. This new algorithm
causes far fewer false hits and still does a good job of detecting
problems when they have really occurred. In particular, this change
should ease the transition to kernels supporting UFS2 which make
some significant changes to the superblock.

Sponsored by: DARPA, NAI Labs

o __P removed
o ansi function prototypes
o unifdef -D__STDC__
o __dead2 on usage prototype
o remove now-bogus main prototype

Remove 'register' keyword.
It does not help modern compilers, and some may take some hit from it.
(I also found several functions that listed *every* of its 10 local vars with
"register" -- just how many free registers do people think machines have?)

Ignore the value of fs_active when comparing superblocks.

Noticed by: "Niels Chr. Bank-Pedersen" <ncbp@bank-pedersen.dk>

Fix a large number of -Wall, -Wformat and -W compiler warnings.
These were mainly missing casts or wrong format strings in printf
statements, but there were also missing includes, unused variables,
functions and arguments.

The choice of `long' vs `int' still seems almost random in a lot
of places though.

Ignore the new superblock fields fs_pendingblocks and fs_pendinginodes
when comparing with the alternate superblock. These fields are used
for temporary in-core information only. This should fix the "VALUES
IN SUPER BLOCK DISAGREE WITH THOSE IN FIRST ALTERNATE" error from
fsck_ffs that has been seen a lot recently.

In fsdb, call sblock_init() which is now necessary to initialise
the global variable dev_bsize. Add a prototype for sblock_init()
to fsck.h, and set the return type correctly.

Minor background cleanups:
1) Set the FS_NEEDSFSCK flag when unexpected problems are encountered.
2) Clear the FS_NEEDSFSCK flag after a successful foreground cleanup.
3) Refuse to run in background when the FS_NEEDSFSCK flag is set.
4) Avoid taking and removing a snapshot when the filesystem is already clean.
5) Properly implement the force cleaning (-f) flag when in preen mode.

Note that you need to have revision 1.21 (date: 2001/04/14 05:26:28) of
fs.h installed in <ufs/ffs/fs.h> defining FS_NEEDSFSCK for this to compile.

Catch up to the dirpref changes by copying new fields in the alternate
superblock from the original superblock so that differences in those new
fields are ignored.

Additions to run checks on live filesystems. This change will not
affect current systems until fsck is modified to use these new
facilities. To try out this change, set the fsck passno to zero
in /etc/fstab to cause the filesystem to be mounted without running
fsck, then run `fsck_ffs -p -B <filesystem>' after the system has
been brought up multiuser to run a background cleanup on <filesystem>.
Note that the <filesystem> in question must have soft updates enabled.

Fsck_ffs did not properly range-check the inode 'di_size'
field, so it was possible for a filesystem marked clean by fsck_ffs
to cause kernel crashes later when mounted. This could occur when
fsck_ffs was used to repair a badly corrupted filesystem.

As pointed out by bde, it is not sufficient to restrict di_size to
just the superblock fs_maxfilesize limit. The use of 32-bit logical
block numbers (both in fsck and the kernel) induces another file
size limit which is usually lower than fs_maxfilesize. Also, the
old 4.3BSD filesystem does not have fs_maxfilesize initialised.

Following this change, fsck_ffs will enforce exactly the same
file size limits as are used by the kernel.

PR: kern/15065
Discussed with: bde
Reviewed by: bde, mckusick

The ffs superblock includes a 128-byte region for use by temporary
in-core pointers to summary information. An array in this region
(fs_csp) could overflow on filesystems with a very large number of
cylinder groups (~16000 on i386 with 8k blocks). When this happens,
other fields in the superblock get corrupted, and fsck refuses to
check the filesystem.

Solve this problem by replacing the fs_csp array in 'struct fs'
with a single pointer, and add padding to keep the length of the
128-byte region fixed. Update the kernel and userland utilities
to use just this single pointer.

With this change, the kernel no longer makes use of the superblock
fields 'fs_csshift' and 'fs_csmask'. Add a comment to newfs/mkfs.c
to indicate that these fields must be calculated for compatibility
with older kernels.

Reviewed by: mckusick

Teach fsck about snapshot files. These changes should have no
effect on operation of fsck on filesystems without snapshots.
If you get compilation errors, be sure that you have copies of
/usr/include/sys/mount.h (1.94), /usr/include/sys/stat.h (1.21),
and /usr/include/ufs/ffs/fs.h (1.16) as of July 4, 2000 or later.

Yesterday I had to fix a badly broken disk, and found that fsck kept dying:

DIR I=64512 CONNECTED. PARENT WAS I=4032
fsck: cannot find inode 995904

fsdb found the inodes with no problem:

fsdb (inum: 64512)> inode 995904
current inode: directory
I=995904 MODE=40777 SIZE=512
MTIME=Feb 14 15:27:07 2000 [0 nsec]
CTIME=Feb 14 15:27:07 2000 [0 nsec]
ATIME=Feb 24 10:31:58 2000 [0 nsec]
OWNER=nobody GRP=nobody LINKCNT=4 FLAGS=0 BLKCNT=2 GEN=38a41386
Direct blocks: 8094568 0 0 0 0 0 0 0 0 0 0 0
Indirect blocks: 0 0 0

The problem turns out to be a program logic error in fsck. It stores
directory inodes internally in hash lists, using the number of
directories to form the hash key:

inpp = &inphead[inumber % numdirs];

Elsewhere, however, it increments numdirs when it finds unattached
directories. I've made the following fix, which solved the problem in
the case in hand.

Submitted by: Greg Lehey <grog@lemis.com>
Reviewed by: Matthew Dillon <dillon@apollo.backplane.com>
Approved by: Kirk McKusick <mckusick@mckusick.com>

Make fsck even more char/blk dev tolerant.

$Id$ -> $FreeBSD$

Cosmetic and documentation changes brought from earlier FreeBSD versions.
(e.g. RCS Id:)

Reviewed by: Don Lewis <Don.Lewis@tsc.tdk.com>
Submitted by: Kirk McKusick <mckusick@McKusick.COM>
Obtained from: Mckusick, BSDI and a host of others

This exactly matches Kirks sources imported under the
Tag MCKUSICK2. These are as supplied by kirk with one small
change needed to compile under freeBSD.

Some FreeBSD patches will be added back, though many have been
added to Kirk's sources already.

Fixed printf format errors.

Correct use of .Nm. Add rcsid. Remove unused #includes. Use err(3).

Reviewed by: dyson@freebsd.org (john Dyson), dg@root.com (david greenman)
Submitted by: Kirk McKusick (mcKusick@mckusick.com)
Obtained from: WHistle development tree

If numdirs is zero, print a helpful message instead of divding by zero later.

Guard against a block size of 0 in the label. When the first
superblock is invalid, fsck looks at the label to help guess where
the next superblock should be. If the partition type is 4.2BSD,
fsck assumed that the block size was valid and divided by it, so
it dumped core if the size was 0.

Initialization of the label was broken almost 3 years ago in rev.1.9
of newfs/newfs.c. Newfs does not change the label at all, so there
is no problem (except the breakage of the automatic search for
backup superblocks) unless something else sets the partition type
to 4.2BSD. However, it is too easy to set partition types to
4.2.BSD by copying an old label or by using a disktab entry to
create the label.

PR: 2537

Kill the Lite2 early "filesystem clean abort" check and go back to
something closer to how we used to do it. The Lite2 way is to check the
"fsclean" flag in the superblock and stop there if so (during preen).
We now do the various superblock sanity checks that we used to do before
since it's cheap. We now get the filesystem state summary again instead
of "FILESYSTEM CLEAN; CHECKING SKIPPED" (or whatever).

Finished (?) merging with Lite2: cleaned up #include mess.

Merge from Lite2. Note that Lite2 has it's own filesystem clean check
skipping code that overrides ours sooner. One should be eliminated,
but for now it works.

Remove trailing whitespace.

Submitted by: phk, added to by bde

Fix all the warnings from `gcc -Wall'.

Don't allow the alternate superblock block number for one file system
to apply to others (except when it is given on the command line).

BSD 4.4 Lite sbin Sources

Note: XNSrouted and routed NOT imported here, they shall be imported with
usr.sbin.