usr.bin: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

usr.bin: Remove ancient SCCS tags.

Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.

Sponsored by: Netflix

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

Restore and document -ps / -nps option.

Sponsored by: Klara, Inc.
Reviewed by: pauamma_gundo.com, pstef, kevans
Differential Revision: https://reviews.freebsd.org/D40788

Fix a few typos in source code comments

- s/procesing/processing/

MFC after: 5 days

indent(1): bug fix after r336333

The bug was that isalnum() is not exactly equivalent to previous code which
also allowed characters "$" and "_", so check for those explicitly.

Reported by: tuexen@

indent(1): rewrite the integer/floating constant scanning part of lexi.c

Remove procedural code that did the scanning, which was faulty and didn't
support complex constants such as 0x1p-61. Replace it with a finite state
machine expressed as a transition table. The table was rewritten by hand
from lx's output, given parts of grammar expressed as regular expressions.

lx is Katherine Flavel's lexer generator, currently available at
https://github.com/katef/libfsm and the parts of grammar were taken from
http://quut.com/c/ANSI-C-grammar-l-2011.html and extended to support binary
integer constants which are a popular GCC extension.

Reported by: bde

indent(1): group global option variables into an options structure

It's clearer now when a variable represents a toggable command line option.

Many options were stored in the parser's state structure, so fix also that.

indent(1): if an identifier can be either a typedef name or as a struct
member, it is taken as the latter

indent(1): improve CHECK_SIZE_ macros

Rewrite the macros so that they take a parameter. Consumers use it to signal
how much room in the buffer they need; this lets them do that once when
required space is known instead of doing the check once every loop step.

Also take the parameter value into consideration when resizing the buffer;
the requested space may be larger than the constant 400 bytes that the
previous version used - now it's the sum of those two values.

On the consumer side, don't copy strings byte by byte - use memcpy().

Deduplicate code that copied base 2, base 8 and base 16 literals.

Don't advance the e_token pointer once the token has been copied into
s_token. This allows easy calculation of the token's length.

indent(1): remove troff output support

The troff output in indent was invented at Sun and the online documentation
for some post-SunOS operating system includes this:
The usual way to get a troffed listing is with the command
indent -troff program.c | troff -mindent

The indent manual page in FreeBSD 1.0 already lacks that information and
troff -mindent complains about not being able to find the macro file.
It seems that the file did exist on SunOS and was supposed to be imported
into 4.3BSD together with the feature, but that has never happened.

Removal of troff output support simplifies a lot of indent's code.

vgrind(1) seems to be a promising replacement.

indent(1): limit character classification functions' input to unsigned char

indent(1): recognize more type names

Most are from C99.

indent(1): don't format function declarations as variables

indent(1): disjoint parser state from lexi()

The function is sometimes used as a look-ahead, so ideally it should bear
no information about parser state.

indent(1): improve predictability of lexi()

lexi() reads the input stream and categorizes the next token. indent will
sometimes buffer up a sequence of tokens in order rearrange them. That is
needed for properly cuddling else or placing braces correctly according to
the chosen style (KNF vs Allman) when comments are around. The loop that
buffers tokens up uses lexi() to decide if it's time to stop buffering. Then
the temporary buffer is used to feed lexi() the same tokens again, this time
for normal processing.

The problem is that lexi() apart from recognizing the token, can change
a lot of information about the current state, for example ps.last_nl,
ps.keyword, buf_ptr. It also abandons leading whitespace, which is needed
mainly for comment-related considerations. So the call to lexi() while
tokens are buffered up and categorized can change the state before they're
read again for normal processing which may easily result in changing
interpretation of the current state and lead to incorrect output.

To work around the problems:
1) copy the whitespace into the save_com buffer so that it will be read
again when processed
2) trick lexi() into modifying a temporary copy of the parser state instead
of the original.

indent(1): remove undocumented and rather useless option (-ps)

It's used to treat the "->" access operator as a binary operator and put
space characters around it.

indent(1): don't indent typedef declarations as object declarations

spdx: initial adoption of licensing ID tags.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D13133

indent(1): don't produce unneeded space character in function declarators.

indent(1): Support binary integer literals.
This was done by Romain Tartière for PR123553. I initially thought that it would break code like this:
#define b00101010 -1
if (0 b00101010)
...

by joining 0 and b00101010 together. However, the real problem with that patch was that once it saw a 0, it assumed that the number was base 2, 8 or 16, ignoring base 10 floating point numbers. I fixed that.

I didn't copy the diagnostic part of the original patch as it seems out of scope of implementing binary integer literals formatting.

PR: 123553
Submitted by: romain (original version)
Approved by: pfg (mentor)

indent(1): Do not define opchar unless it will be used.

"opchar" is only used once depending on "undef" macro being defined.
Conditionalize it in the same way.

Submitted by: Piotr Sephaniak

indent(1): Fix indent's confusion about custom FreeBSD macros.

Teach indent(1) about storage-class specifiers. Don't assume
"in_parameter_declaration" state if "in_decl" hasn't been set. Don't set
"in_decl" for storage-class specifiers.

That set of changes helps with recognizing the difference between file
scope declarations like this:

static LIST_HEAD(, alq) ald_active;
static int ald_shuttingdown = 0;
struct thread *ald_thread;

and old style function declarators like this:

static int
do_execve(td, args, mac_p)
struct thread *td;
struct image_args *args;
struct mac *mac_p;
{

Unfortunately, at the same time this change makes indent(1) require
explicit int in declarations like "static a;", in order to understand that
it's part of a declaration. On the other hand, declarations like in the
first example are no longer indented as if ald_shuttingdown and ald_thread
were parameters of a function named LIST_HEAD.

Submitted by: Piotr Stefaniak

indent(1): Properly handle the wide string literal and wide char constant L.

indent(1) treated the "L" in "L'a'" as if it were an identifier and forced
a space character after it, breaking valid code.

PR: 143090
MFC after: 2 weeks

indent(1): Fix off-by-one in control flow leading dead code.

Coverity correctly reported that it's impossible for /comparison/ to be 0
here, because the only way for the for loop to end is by /comparison/
being < 0.

Fortunately the consequences of this bug weren't severe; for duplicated
entries in the typedef names file it would unnecessarily duplicate strings
with strdup(), but pointers to those would replace existing ones. So this
was a memory leak at worst.

CID: 1361477
Obtained from: Piotr Stephaniak

indent(1): Fix memory leaks pointed out by clang-analyzer.

Shift the responsibility of allocating memory for the string duplicate
from the caller (set_option, add_typedefs_from_file) to the callee
(add_typename) as it has more knowledge about when the duplication
actually needs to occur.

Taken from: Piotr Stefaniak

indent(1): Use bsearch() for looking up type keywords.

Reference:
https://github.com/pstef/freebsd_indent/commit/f3b8e6e57fd47364b2360b44011a2fad5d11adc7

Submitted by: Piotr Stefaniak
Differential Revision: https://reviews.freebsd.org/D6966

indent(1): add new -sac and -U options.

Add -sac (space after cast) and -nsac options.
These control whether space character is put after a cast operator or not.
Default is -nsac.

Add -U option for providing a file containing list of types.
This is needed for properly deciding which asterisks denote unary
operation and which denote binary.

These come from PostgreSQL.

Reference:
https://github.com/pstef/freebsd_indent/commit/84b00e3d46dfd6d955b2f481a1f3b275de9ad6d1
https://github.com/pstef/freebsd_indent/commit/49c52cf383fa2a246a1a22c6640a5a21b0f1fd90

Differential Revision: https://reviews.freebsd.org/D6966 (Partial)
Submitted by: Piotr Stefaniak

indent(1): accept offsetof(3) as a keyword.

Reference:
https://github.com/pstef/freebsd_indent/commit/c470e5e2c974aa38450ca4762b93829f7a7bfa4d

Differential Revision: https://reviews.freebsd.org/D6966 (Partial)
Submitted by: Piotr Stefaniak

indent(1): Use a dash in the license headers.

Use of the canonical dash avoids indent(1) from reformatting the
license headers.

indent(1): Yet more style issues.

strchr(3) returns a pointer not a boolean.
Attempt to make the style somewhat more ocnsistent with what indent
had before recent changes.

Pointed out by: bde

indent(1): Use NULL instead of zero for pointers.

indent(1): Support "f" and "F" floating constant suffixes.

Actually this just brings back r303487 with the correct commit log.

Differential Revision: https://reviews.freebsd.org/D6966 (Partial)
Obtained from: Piotr Stefaniak

Revert r303487: Wrong commit log.

Will be brought back with the correct log.

indent(1): fix struct termination detection.

Small style cleanup while here.

Differential Revision: https://reviews.freebsd.org/D6966 (Partial)
Obtained from: Piotr Stefaniak

indent(1): Fix typo.

It's typedef, not typdef.

Obtained from: OpenBSD (CVS rev. 1.20)
MFC after: 3 days

A new jail(8) with a configuration file, to replace the work currently done
by /etc/rc.d/jail.

MFC r206687: indent(1): don't treat bare '_t' as a type name with -ta

indent(1): don't treat bare '_t' as a type name when -ta is used

It seems that identifier "_t" is sometimes used as a variable name,
even in our tree. Not that I endorse that, but still it's better
to require at least one character before _t suffix to consider
an identifier to be a type name.

Reported by: Alex Vasylenko <lxv@omut.org>
MFC after: 1 week

MFC r205989: indent(1): new option, -ta, to treat all *_t identifiers as types

indent(1): add a new option, -ta, to treat all *_t identifiers as types

Submitted by: Hans Petter Selasky
Discussed with: bde
MFC after: 10 days

Create the altix project branch. The altix project will add support
for the SGI Altix 350 to FreeBSD/ia64. The hardware used for porting
is a two-module system, consisting of a base compute module and a
CPU expansion module. SGI's NUMAFlex architecture can be an excellent
platform to test CPU affinity and NUMA-aware features in FreeBSD.

- Import the HEAD csup code which is the basis for the cvsmode work.

Use the appropriate error function for displaying the error,
instead of printing it to stdout.

MFC after: 1 week

Fixed misformatting of "struct foo *bar" in function parameter lists. It
was mangled to "struct foo * bar". There should be an option to control
this, but no space is normal. This finishes fixing the bugs in rev.1.4.

indent(1) still doesn't really understand types in parameter lists. It
thinks keywords inside parentheses are for casts or sizeofs. This works
accidentally for scalar types and this quick fix makes it work similarly
but not so accidentally for struct/union/enum types.

Expanded the size of the keyword table from 100 to 1000 entries so that
the number of typedef-names is not so limited. Same as in rev.1.4.

Added the "const" and "volatile" to the keyword table. Rev.1.4 added
these but they were misclassified so they were not formatted as types.
indent still doesn't really understand them. E.g., it mangles
"char * const *foo" and "char *const *foo". This change mainly stops
it mangling "char const foo" to "char<declaration-indent>const foo".

Backed out rev.1.4 and 1.7 so they they can be implemented and committed
properly. Of the 3 changes mentioned in the log message for rev.1.4,
the first (implementing -[n]fcb) was correct but didn't touch this
file, the second (no-space-after-sizeof) was not actually done (it is
the default and is controlled by the undcoumented -[n]bs options), and
the third (no-space-after 'struct foo *') was very buggy and was reduced
to wrong comments and other style bugs by backing out the main part
of it in rev.1.6. Rev.1.4 had 2 changes which were not mentioned in
its commit log: expand specials[] so that more than -83 typedef-names
can be specified (this was the one working change in rev.1.4), and add
"const" and "volatile" to specials[] (this was buggy).

err() on allocation failure. WARNS=9 compliant
use #if 0, #ifndef lint, #endif /* not lint */, #endif ordering
when a message is provided, use errx() instead of err().

Spelling

Consistently use FBSDID

Remove deprecated register qualifier.

1) Const enough things to avoid warnings.
2) Cast ifdef_level to a size_t before comparing it to a ratio of size_ts.
Ifdef_level should always be positive.
3) Complete prototype for chfont.
4) Cast some ptrdiff_ts to ints before using as a field width.
5) Avoid shadowing a local variable p with another local variable p.

Make this compile cleanly when warnings are enabled:
- ANSIfy function declarations
- braces around initializers structs within structs
- add parens in complicated expressions
- disambiguate dangling elses
- no more implicit int
- make functions static where possible
- use prototypes
- don't use varargs hack for diag()

Requested by: joerg
MFC after: 2 weeks

Properly handle backslash newline within an identifier or keyword.

PR: bin/6015
Submitted by: myself (schweikh)
Patch by: Alexey V.Neyman <alex.neyman@auriga.ru>
Tested by: indenting my chess problem solver and running its test suite
MFC after: 3 weeks

Fix damage I did in rev 1.4 that broke formatting this example:

void foo(struct bar *bar) { return 0; }

Don't abuse the SCCS `@(#)' for RCS.

Requested by: bde

Add or fix FreeBSD IDs.

"Implement -[n]fcb (formatting of block comments) and attempt to implement
no-space=after-sizeof (not optional) and no-space-after 'struct foo *'
(not optional). Without these, indent unKNFizes even more perfectly KNF code."

Submitted by: bde

Support integer constant unsigned-suffix and long-long-suffix.

Remove trailing whitespace.

BSD 4.4 Lite Usr.bin Sources