lint: accept C23 [[fallthrough]] statements

Needed by bind9/getaddresses.c:137.

lint: remove unused flag '-u' from lint1

This flag is completely handled by lint2.

lint: in strict bool mode, allow do-while-0

Even though 0 is not a boolean constant, allow this common idiom, to
help in those cases where the C preprocessor used by lint does not mark
tokens as coming from system headers (Clang).

lint: move code for parsing balanced token sequences to lex.c

To access yytext from lex.c, the lexer needs to be in %pointer mode,
which was specified in IEEE Std 1003.1, 2004 Edition and thus should be
old enough to be available on platforms other than NetBSD, for use in
tools/lint1 where lint1 is built before yacc and lex.

lint: parse but otherwise ignore C23 attributes

The C23 attributes are only parsed before an expression in an expression
statement, as a proof of concept. Other places will follow later.

lint: in C23 mode, support the nullptr constant

lint: measure the alignment in bytes, not bits

While measuring the alignment in bits makes sense when building a struct
type with bit-fields, in all other places it is more confusing than
helpful.

The only visible change is that in debug mode, the format of type sizes
and alignment changed. Since the size of all complete types is a
multiple of a byte (as defined in the C standard), sizes and alignments
are reported in bytes as well. Only while a struct is being built, the
type size may include an additional '+x' for the bits of a bit-field.

lint: support _Alignas and __attribute__((__aligned(4)))

lint: clean up and speed up the check for snprintb

lint: clean up

lint: clean up

lint: use fewer struct keywords

lint: check for unreachable bits and fields in snprintb formats

While here, clean up a few existing checks.

lint: test format strings from snprintb calls

The functions snprintb and snprintb_m are specific to NetBSD, and their
format strings are tricky to get correct. Provide some assistance in
catching the most common mistakes.

lint: make function call arguments directly accessible

Previously, the arguments of a function call expression were arranged in
a linear tree structure, from right to left. To allow easier access to
the arguments, store them in an array instead.

lint: keep strings in their source representation

This allows further analysis depending on whether individual characters are
escaped as octal, hexadecimal or not at all.

lint: clean up comments, reduce scope of variables

lint: use standard buffer for storing string values

No functional change.

lint: rename symt_t to symbol_kind

It was confusing to have two kinds of "symbol type" (s_type and s_symt),
so rename all related identifiers to be more distinctive.

No functional change.

lint: correctly set system-header flag on cast-expression

When a cast-expression comes partly from a system header, determine at
the ')' whether the whole cast-expression comes from the system header.
Previously, it was based on the operand, which contradicted the
documentation of tn_sys.

Mainly affects strict bool mode (where expressions from system headers
are handled more leniently), as well as query 9 for parenthesized return
expressions.

Discovered upon manual inspection, as calling expr_alloc_tnode should
never be necessary when creating an expression node with operands;
there's build_op for that purpose.

lint: allow complex offsetof(type, member-designator)

Both GCC 11 and Clang 8 accept member-designators that are not
identifiers but designator sequences, such as in 'offsetof(struct stat,
st_atim.tv_sec)', so make lint accept them as well.

lint: allow querying for 'static' followed by non-'static' declaration

lint: in declarations, replace tab with space

Previously, in some cases, the keywords 'const' or 'struct' were
followed by a tab, which doesn't make sense.

No functional change.

lint: remove unused string literals

lint: fix handling of unnamed function parameters

lint: distinguish between arguments and parameters

lint: improve debug logging for declarators

lint: in debug mode, default to indenting the debug log

Only the 'parsing' lines are not indented, as line breaks are
independent from the structure of the code.

lint: condense code for ending a function

No functional change.

lint: constify

lint: use standard function attribute for noreturn functions

lint: extract 'vararg' and 'prototype' flags from global 'dcs'

These flags are only relevant for parameter lists, so add a separate
type for it.

No functional change.

lint: clean up duplicate code for finding struct/union members

lint: fix use-after-free bug in GCC statement expressions

lint: add detailed logging for finding memory allocation bugs

lint: use separate types for parsing pointer types and type qualifiers

The documentation of the previous 'qual_ptr_t' was misleading, as that
type was used for other type qualifier lists as well, even if these were
not related to pointer types.

lint: indent copyright lines consistently

lint: reduce code size for handling lint comments

lint: clean up code related to lint comments

Remove type information from variable names, as the word 'flag' did not
indicate that the variables were negated.

Remove contradicting comments. Suppressing a warning for 'this line',
'this and the following line' and 'this statement or declaration' cannot
all be accurate at the same time.

lint: don't treat 'inline' as a storage class

The additional conflict in the grammar is in the same place as the other
conflicts, as T_SCLASS and T_FUNCTION_SPECIFIER are now separate tokens.

No functional change.

lint: warn about function definitions without header declaration

The existing warning was only issued for function declarations, not for
function definitions.

The interesting change in the tests is in msg_351.c. Many other tests
use non-static functions due to their syntactic brevity. In these
tests, the warning is disabled individually, to allow new functions to
be added without generating warning 351.

lint: only skip 'unused' warnings after errors, not other warnings

Previously, in -w mode, any warning suppressed further 'unused'
warnings, even though there was no need to do that. This can be seen in
the test gcc_attribute_var.c, where only the last unused variable from a
function was marked as unused, the others slipped through.

Fixed by counting the errors and the warnings separately and only
combining them if actually desired.

lint: add initial support for C23

Required by xsrc/external/mit/MesaLib.old, brw_eu_validate.c, which
initializes a struct using empty braces: 'return (struct string){};'.

lint: rename 'quad' to 'signed int' or 'unsigned int'

No functional change.

lint: extend debug logging for declaration levels

Indent the debug logging according to the declaration level.

Since there are a few cases where the enclosing declaration levels are
modified, log the whole declaration level stack whenever a level begins
or ends.

lint: clean up duplicate and dead code for integer constants

No functional change.

lint: fix initialization of unnamed union member

lint: clean up names related to declaration levels

The previous prefix 'DK_' (declaration level kind) had a conflict with
the 'DK_' (designator kind) in init.c, so change the prefix to 'DLK_'.
The new name for dinfo_t is decl_level, which is more expressive.

No functional change.

lint: fix handling of unnamed struct/union members

The support for unnamed struct/union members that was added in decl.c
1.60 from 2015-10-13 was simple but wrong. It didn't cover initializers
of these structures and computed wrong sizes for structures containing
anonymous unions. At that time, the handling of initializers was broken
as well, it was fixed 6 years later in init.c 1.229 from 2021-12-22.

Real-life examples for code that lint couldn't handle are:

* external/bsd/jemalloc/dist/src/jemalloc.c
* external/mit/xorg/lib/dri.old/Makefile

lint: clean up handling of declarations

No functional change.

lint: clean up function names

No functional change.

lint: clean up completion of struct, union and enum declarations

No functional change.

lint: remove redundant 'extern' from function declarations

No binary change.

lint: be strict when parsing command line for excluded message IDs

Previously, lint accepted -X '1, 2, 3', while the manual page lists the
IDs without spaces.

On 32-bit platforms, lint accepted -X -4294967295, and on 64-bit
platforms, it accepted the corresponding larger numbers.

The code for parsing message IDs and query IDs conceptually does the
same, but the implementations differed for no reason.

lint: in debug mode, use different names for macro and function

No functional change.

lint: warn about extern declarations outside headers

https://mail-index.netbsd.org/tech-userlevel/2023/03/15/msg013727.html

lint: remove unused internal_error

lint: eliminate unnecessary indirection

No functional change.

lint: rename functions for function definition

No functional change.

lint: remove custom memory allocator

Besides adding complexity, the custom memory allocator didn't invalidate
freed memory, which made it harder to find possible use-after-free bugs.

lint: reduce pointer indirection for unnamed entities

No functional change.

lint: inline previous_declaration for nonconstant message IDs

This adds back the compile-time printf validation in debug mode that was
missing before.

lint: clean up visual clutter

No functional change.

lint: rename functions to be clearer

No need anymore to keep external identifiers at the "6 significant
initial characters" mandated by C90.

lint: rename dcs manipulation functions to be clearer

No functional change.

lint: add additional queries that are not enabled by default

In the last 18 months, several lint warnings have been made adjusted to
allow common usage patterns. For example, lint no longer warns about a
constant condition in the statement 'do { ... } while (false)' (message
161), as this pattern is well-known in statement-like macros, making it
unlikely that the 'false' is a mistake. Another example is casts
between unequal pointer types (message 247) for a few well-known
patterns that are unlikely to be bugs.

Occasionally, it is useful to query the code for patterns or events that
would not justify a warning. These patterns are modeled as predefined
queries that can be selected individually, in addition to and
independently of the existing warnings and errors.

New queries can be added as needed, in the same way as new warnings.
Queries that are deemed no longer used can be deactivated in the same
way as warnings that are no longer used.

As long as none of the queries is enabled, they produce a minimal
overhead of querying a single global variable. Computations that are
more expensive than a few machine instructions should be guarded by
any_query_enabled.

https://mail-index.netbsd.org/source-changes-d/2022/06/28/msg013716.html

ok christos@

lint: add debug logging for struct and enum details

lint: move error handling code from main1.c to err.c

No functional change.

lint: rename mod_t.m_requires_bool to m_compares_with_zero

The operators NOT, LOGAND, LOGOR and QUEST only require _Bool in strict
bool mode, in default mode they accept any scalar expression and compare
it with zero. The new names are more accurate.

No functional change.

lint: unexport lint's messages

There was only a single place where a message was used directly, instead
of calling one of the several functions for this purpose.

lint: inline macro 'sflag'

Mark all places where lint's C90 mode is stricter than its C99 mode.

Most of the situations in which lint produces only warnings instead of
errors covered by the "Constraints" sections in the relevant standards.
This doesn't prevent a specific compiler from accepting it though.

No functional change.

lint: inline macro 'tflag'

The definition of the macro tested both allow_trad and allow_c90, but
there is only a single mode in which allow_c90 is false, therefore it
suffices to test only that.

While double-checking each occurrence of tflag individually, I learned
why lint performs lookups of struct members only by name, independently
of the struct in which they are declared. See typeok_arrow for details.

No functional change.

lint: document why C11 keywords are available in C99 mode as well

lint: inline macro c11flag

No functional change.

lint: migrate gflag to allow_gcc

No functional change.

lint: model C language levels in a future-compatible way

The options -t, -s and -S are confusing because they are used
inconsistently. The option -S enables C99 features, but when using it
instead of -s, it also doesn't enable all checks required by C90 and
later. Prepare fixing of these inconsistencies by replacing the flag
variables with language levels that can be extended in a
straight-forward way as new C standards arrive.

| option | allow_trad | allow_c90 | allow_c99 | allow_c11 |
|--------|------------|-----------|-----------|-----------|
| -t | x | - | - | - |
| (none) | x | x | - | - |
| -s | - | x | - | - |
| -S | - | x | x | - |
| -Ac11 | - | x | x | x |

Each usage of the old flag variables will be inspected and migrated
individually, to clean up the subtle variations in the conditions and to
provide a simpler model.

When lint was created in 1995, its focus was migrating traditional C
code to C90 code. Lint does not help in migrating from C90 to C99 or
from C99 to C11 since there are only few silent changes, and simply
because nobody took the time to implement these migration aids. If
necessary, such migration modes could be added separately.

There is a small functional change: when the option -s is combined with
either -S or -Ac11, lint now only keeps the last of these options.
Previously, these options could be combined, leading to a mixture of
language levels, halfway between C90, C99 and C11. Especially combining
traditional C with C11 doesn't make sense, but xlint currently allows
it.

The 3 tests that accidentally specified multiple language levels have
been adjusted to a single language level.

lint: distinguish between storage class and declaration kind

These types overlap but are not the same.

No functional change.

lint: rename length to length_in_bits

No functional change.

lint: inline setcomplete

No functional change.

lint: add debug logging for the declaration stack

To track down the wrong edge cases in decl_direct_abstract.c.

lint: rename function 'cleanup' to be more expressive

No functional change.

lint: add debug logging for symbols and the symbol table

This logging is not active by default, the functions debug_sym and
debug_symtab can be called as needed during a debug session.

lint: encode lifetime of allocated memory in function names

No functional change.

lint: encode lifetime of allocated memory in the function names

No functional change.

lint: fix memory corruption in statement expressions (since 2021-12-17)

The commit that introduced the assertion failure looks innocent, it only
adds a few predefined functions for GCC mode. Nevertheless, before that
commit, lint consistently complained about 'error: void type illegal in
expression [109]', which doesn't make sense either.

This fix also removes the creative use of the initialization stack to
store the type of the statement expression. Having a separate stack for
these statement expressions makes the code easier to understand.

lint: extract code for handling statement expressions from the grammar

This prepares the fix of the memory corruption bug that is demonstrated
in t_integration.sh, test case assertion_failures.

No functional change.

lint: rename debug_indent to debug_print_indent

The previous name could be mistaken to mean "increase the indentation of
the debug output". Instead, the function prints the current indentation.

In externs1.h, the macro definition was a duplicate, the macros were
sorted differently than the functions a few lines above.

No binary change.

lint: add grammar rule for the beginning of a designation

This will be necessary to properly implement handling of initializers
and braced initializer-lists.

No functional change for now since the designation is already reset
after each expression and '}'. To handle initializations properly, the
designation must not be reset after each expression, it must advance to
the next member instead.

lint: narrow down parameter of build_name

Passing an arbitrary tokenizer symbol left too much freedom and
uncertainty to the caller, and 0 was a magic number in this context.

No functional change.

lint: unexport struct_or_union_member

Strangely, lint2 didn't notice this unnecessary export.

lint: fix check for function calls in strict bool mode

Previously, if a function call occurred in the controlling expression,
its return type could be any scalar, not just bool. This was against
the goal of strict bool mode, which makes bool a separate and
incompabile type to all other types. For example, it would allow
controlling expressions like 'strcmp(a, b)' without the usual '!= 0',
but only if at least one of 'a' and 'b' came from a macro definition
from a system header.

The fix is that the decision of whether the type of the controlling
expression may be scalar is no longer based on the operand types but on
the main operator of the controlling expression.

lint: rename attron to in_gcc_attribute

No functional change.

lint: remove unused function for generic messages

The function message_at is still used, for information about previous
occurrences of a symbol.

lint: add __sync_ and _mm_ as prefixes for builtin functions

These two additions cover all cases that occur in the current NetBSD
build on x86_64. This allows build_name to use the usual pattern 'if
Sflag then error else if sflag then warning'. That function currently
issues a warning in C99 as well, even though C99 prohibits implicit
function declarations.

lint: do not emit GCC builtin functions

Lint1 no longer emits declarations of GCC builtin functions and calls to
them.

Previously, lint generated 3421 useless warnings in a default NetBSD
build, like this:

__atomic_load_n, arg 1 used inconsistently
acl.c(216)[pointer to unsigned int]
rbtdb.c(921)[pointer to unsigned short]

This was because lint just doesn't understand that these functions are
type-generic, which is indeed unusual in C.

These useless warnings made the lint output more frightening than it
should actually be. Together with the strange formatting of the
diagnostics (space-space-tab after the main message, two spaces and two
colons between the occurrences, symbols are listed in hashcode order),
this creates the impression that lint is not intended to be a
user-friendly tool.

For now, fix the excess warnings, leaving the other items for later.

lint: use 'unsigned int' for bit-size of types in convert_integer

There was no need to have two separate magic values (0 and -1) to mean
the same.

No functional change.

lint: use 'unsigned int' for bit-size of types

Lint does not need to support any types larger than 256 MB since they
don't occur in practice. Practically, such large types have never been
supported at all since the function type_size_in_bits used int for the
internal calculations, resulting in overflows.

lint: fix a few lint warnings about type conversions

A simple 'unsigned int' is more than enough for representing the size of
a bit-field, as well as the maximum alignment of any type.

No functional change.

lint: inline msb

This reduces the binary size on x86_64 by about 500 bytes.

No functional change.

lint: remove unnecessary width parameter for msb

No functional change.

lint: remove Tflag and pflag from common declarations

These flags are only needed by lint1 and lint2, but not by the driver
xlint.

No functional change.

lint: change return type of 'msb' from int to bool

No functional change.

lint: merge duplicate debugging code

The functions 'debug_node' and 'display_expression' were similar enough
to be merged.

Migrate debug_node to use the existing debug logging functions.

Remove the now unused option 'd' from the options string.

lint: remove option -d, clean up debug logging

The command line option -d was not used by /usr/bin/lint, and it only
triggered a handful of debug messages. Move this debug logging over to
the compile-time -DDEBUG setting.

Move display_expression further up to avoid the forward declaration.

lint: do not evaluate arguments of debug_step

To analyze the unexpected test failure of op_shl_lp64, I had reverted
debug_step to evaluate its arguments. I then accidentally committed
that without running the tests again.

Anyway, the previous commit can now be used as a demonstration that
initdecl is indeed missing the initialization for __uint128_t, which
leads to the internal error in op_shl_lp64.

lint: clean up debug logging

The calls to debug_step, unlike printf, don't need a trailing newline.

Remove the debug_step0 macro and its relatives since lint already uses
enough other features from C99 that it essentially requires this
standard, which supports varargs macro arguments. Among these features
are __func__ and printf("%zu").

In non-debug mode, do not evaluate the arguments of debug_step.
Evaluating the arguments had caused an internal error when running the
test op_shl_lp64. This is indeed a bug since initdecl should have
initialized the type table for __uint128_t. This had been forgotten
when support for __uint128_t was added in decl.c 1.69 from 2018-09-07.

No functional change.

lint: extract debug logging to separate file

Lint currently has several different kinds of debug log:

* The -DDEBUG log is controlled at compile time.
* The -d command line options enables some other debug logging.
* The -DYYDEBUG log for parsing is controlled at compile time.
* The -y command line option only has an effect in -DYYDEBUG mode.

Extracting the logging into a separate file is a first step towards
unifying these logs and making the code for debug logging stand out less
than the current #ifdef DEBUG.

No functional change.

lint: add debugging output for the grammar tokens

No functional change outside debug mode.

lint: merge duplicate code for generating unqualified type

This is a preparation for fixing the wrong warnings in msg_115.c.

No functional change.

lint: move scl_name from print.c to decl.c

It is only needed there.

lint: use consistent naming scheme for functions that build nodes

No functional change.

lint: split 'build' into build_binary and build_unary

No functional change.

lint: replace call to ttos with type_name

Since tyname.c 1.20 from 2021-01-02, type_name is as simple to use as
ttos and more expressive. It can also be called multiple times without
invalidating the returned strings.

Used only in debug mode.

lint: extract build_member_access from the grammar

No functional change.

lint: rename clrtyp/deftyp to begin_type/end_type

The abbreviations clr/def did not make it obvious that these two
functions or grammar rules form pairs.

No functional change.

lint: rename type generic_association_types to generic_association

The word 'types' was misleading and unnecessary.

No functional change.

lint: rename xsign to convert_integer

The term sign-extend was too specific, the function actually does a
broader conversion.

No functional change.

lint: rename pqinf_t to qual_ptr and clean up code

The 'inf' from the type name meant 'information' and was redundant. Each
object of that type represents a single pointer level, which made the
documentation about 'pointers' a bit confusing.

The members of struct qual_ptr are now in the canonical reading order,
which is 'const volatile pointer'.

No functional change.

lint: fix result type of _Generic expressions

lint: remove unused print_tnode

The function display_expression serves the same purpose and prints more
details.

lint: in debug mode, print name of the "storage class"

This helps in tracking down the internal errors related to declarations,
found by running afl.

lint: replace array access with function calls

First and foremost, the test d_c99_complex_split accessed the array
qlmasks out-of-bounds, with an index of 128 for the type 'double
_Complex'. This invoked undefined behavior since the maximum allowed
index was 64.

Replacing the raw array accesses with function calls allows for bounds
checks to catch these errors early.

Determining the value bits for a 'double _Complex' does not make sense
at all since it is not an integer type. This means that lint didn't
handle these types correctly for several years. Support for int128_t
has been added in inittyp.c 1.12 from 2018-09-07, support for _Complex
has been added in inittyp.c 1.9 from 2008-04-26.

Determining the value bits for an int128_t would make sense, but the
unit tests don't contain examples for this type since at the moment all
unit tests must produce the same results on 32-bit and 64-bit platforms,
and the 32-bit platforms don't support int128_t.

lint: pass pos_t via const pointer

Thanks for the suggestion, christos@.

lint: add error_at, warning_at, message_at

Right now there are several places in the code that use the global
variable curr_pos for passing a parameter to the diagnostic functions.
That's not what global variables are for.

Make it easy for the code to migrate to the parameter-passing style.

No functional change.

lint: do not allow the diagnostics to be modified

lint: add option to accept C11 features

The list of available letters for the command line options gets shorter
and shorter. Most of the interesting letters are already used for some
warning categories. Curiously, -A, -W and -E were all still available.

The option -A nicely matches the intention of the option, which is to
allow a certain set of language features. To keep the option available
for further extensions, define -Ac11 as the currently only valid option
of that kind. This allows straight-forward extension for C17 and future
language standards, as well as independent feature-sets. The options -W
and -E may someday complement the -A option, using the allow/warn/error
categories.

lint: add support for C11-isms such as int[static 3]

lint: clean up handling of preprocessing output lines

No functional change.

lint: in code from included files, print stack trace

Previously, the standard NetBSD build generated several lint warnings in
lhash.h from OpenSSL, without providing any hint as to which file
actually included that header. In cases like these, lint now interprets
the line number information in the preprocessor output from GCC to
reconstruct the exact include path to the file in question.

The program check-expect.lua had to be rewritten almost completely since
it assumed that all diagnostics would come from the main file. In all
existing tests, this was true, but these tests did not cover all cases
that occurred in practice. Now it records the complete location of the
diagnostic instead of just the line number.

lint: move check for strict bool mode into separate file

No functional change.

lint: warn about for wrong type cast in argument to ctype.h functions

The argument to most of the functions from <ctype.h> "shall either be
representable as an 'unsigned char' or shall equal the value of the
macro EOF".

When confronted with the infamous warning 'array subscript has type
char', there are enough programmers who don't know the background of
that warning and thus fix it in a wrong way. Neither GCC nor Clang
explain its warning to target these programmers.

Both GCC and Clang warn about 'array subscript has type char', but they
ignore the other requirements of the <ctype.h> functions, even though
these are in the C standard library.

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94182
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95177
https://stackoverflow.com/a/60696378

lint: rename functions for duplicating types

No functional change.

lint: name memory allocation functions consistently

No functional change.

lint: rename getnode to expr_zalloc_tnode

The new name highlights that the returned memory is only valid in the
scope of the current expression. This was misleading before since the
other related functions all have a 't' (probably for 'temporary') in
their names.

Also encode in the function name that the returned memory is zeroed out
as that could not be inferred from the old name.

No functional change.

lint: rename mbl to memory_block

No functional change.

lint: rewrite handling of initializations, fixing several bugs

The previous implementation had a wrong model of how initialization
happens in C99, its assertions failed in all kind of edge cases and it
was not possible to fix the remaining bugs one at a time without running
into even more obscure assertion failures.

The debug logging was detailed but did not help to clarify the
situation. After about 20 failed attempts at fixing the small details I
decided to start all over and rewrite the initialization code from
scratch. I left the low-level parts of handling designators, the code
that is independent of brace_level and the high-level parts of how the
parser calls into this module. Everything else is completely new.

The concept of a brace level stays since that is how C99 describes
initialization. The previous code could not handle multi-level
designations (see d_init_pop_member.c). There are no more assertion
failures in the initialization code.

Some TODO comments have been left in the tests to keep the line numbers
the same in this commit. These will be cleaned up in a follow-up
commit.

The new implementation does not handle initialization with "missing"
braces. This is an edge case that both GCC and Clang warn about, so it
is not widely used. If necessary, it may be added later.

The new implementation does not use any global variables in the vast
majority of the functions, to make all dependencies and possible
modifications obvious.

lint: sprinkle const on function declarations

No functional change.

lint: rename public designator functions

Their previous names were too similar to the actual implementation
functions, which was confusing.

No functional change.

lint: rename decl1ext, reduce visibility

No functional change.

lint: rename LERROR to INTERNAL_ERROR

The '#ifndef' in tyname.c is meant to distinguish between lint1 and
lint2, it is not meant to be defined from anywhere outside the lint code
itself.

No functional change.

lint: rename filename management functions

No functional change.

lint: rename fnaddreplsrcdir to something less cryptic

No functional change.

lint: remove unused fnalloc

lint: rename pushctrl and popctrl to be more expressive

While here, remove the magic number 0 that in this context means
"function body".

No functional change.

lint: rename pushdecl and popdecl to be more expressive

The previous names were highly ambiguous. The 'decl' could have meant
'declaration', which would be the usual abbreviation. It could also be
split into 'dec' and 'l', meaning 'declaration level', which would make
more sense in this particular context.

To avoid having to guess anything about these names, rename the
functions. Instead of 'push' and 'pop', I renamed them to 'begin' and
'end' since these are the high-level operation that are of interest.
That the hierarchy of declaration levels is implemented as a stack is
nice to know but not as important to understand the whole situation.

No functional change.

lint: fix names of functions dealing with designations

These functions modify the whole designation, not only a single
designator.

No functional change.

lint: fix wrong error message about type mismatch in compound literal

Now that the code contains explicit markers for starting and ending an
initialization, and having the guarantee that an assertion fails
whenever some code accesses the state of the "current initialization"
even though there is no ongoing initialization gives me much more
confidence in the correctness of the code. The calls to
begin_initialization and end_initialization always appear in pairs,
enclosing the minimal amount of code necessary for initialization.

In a nutshell, global modifiable state is error-prone and hard to
understand.

A nice side effect is that the grammar no longer needs a special rule
for the outermost initializer since the functions for the debug logging
are now called explicitly.

The code that misuses the initialization state just because it needs to
temporarily store a sym_t somewhere is now clearly marked as such. A
GCC statement expression can appear anywhere and is therefore
independent of the initialization. Most probably the code can simply
refer to the local variable in the grammar rule itself, or this variable
needs to be encoded in the grammar %union. For sure there is a better
way to handle this.

There is no longer a need that the function 'declare' initializes the
initialization state, it was just the wrong place to do this.

lint: add indirection for accessing the current initialization

This indirection will be needed to handle nested initializations, which
are a new feature of C99. These are currently not handled correctly,
see msg_171.c.

No functional change.

lint: invert 'rchflag', call it warn_about_unreachable instead

No functional change.

lint: rename functions for handling control statements

No functional change.

lint: rename ftflg to seen_fallthrough

One less abbreviation to remember when reading the code.

No functional change.

lint: use standard form of statement macro for debug_node

No functional change.

lint: rename tsize to type_size_in_bits

The shorter name size_in_bits was already taken by the function-like
macro with argument type tspec_t.

No functional change.

lint: rename push_member and pop_member

These two functions are supposed to model the designator that is used
for initializing structs and arrays. The implementation is still buggy
and does not work at all for C99 designators with multiple names, see
d_init_pop_member.c.

For now, just rename the functions to head in the right direction.

No functional change.

lint: add debug logging for C99-style initialization of arrays

No functional change outside debug mode.

lint: move to_int_constant from the grammar to decl.c

This way, the code is covered by running 'make lint'. The code from the
grammar is not covered, therefore it still uses int instead of bool in a
few places.

Inline the comparison functions for uint64_t. These functions didn't
add any clarity to the code.

No functional change.

lint: move main part of idecl over to decl.c and rename it

No functional change.

lint: rename 'blklev' to 'block_level'

No functional change.

lint: rename getbound to alignment_in_bits

No functional change.

lint: extract check_non_constant_initializer from init_using_expr

No functional change.

lint: fix definition of debug_node in non-debug mode

lint: indent node details in debug mode

lint: extend debugging for initializing objects

No functional change outside debug mode.

lint: rename mkinit to init_using_expr

No functional change outside debug mode.

lint: warn about mismatch in getopt handling

lint: don't warn about constant condition in 'do { } while (0)'

lint: rename incompl to is_incomplete

No functional change.

lint: expand abbreviations in lexer function names

No functional change.

lint: apply strict bool mode to lex.c

There are 2 remaining expressions:

In line 244, !(kw->kw_deco & deco) is a bitwise and. This is already
allowed for enums, it needs to be allowed for arbitrary integer
expressions as well. This covers the many places where plain integers
are used for bit fields, together with #define. This pattern is not as
typesafe as using enums, still it is common practice.

In line 769, the expression !finite(f) is a legitimate use of a function
that has return type int for traditional reasons. It's the same as for
ferror.

There are several other functions like unlink, open or strcmp that have
return type int as well, but with a different meaning. It is not yet
clear what the best way is to handle these different meanings. Having
to write finite(f) == 0 in strict bool mode doesn't look idiomatic, on
the other hand, !strcmp(s1, s2) is exactly the pattern that strict bool
mode wants to avoid.

lint: fix two wrong error messages in strict bool mode

The strict bool mode gets complicated because for system headers the
rules need to be relaxed since they cannot be changed easily, often not at all.

Still, if lint validates a program in strict bool mode, that program
must run with equal behavior regarding boolean expressions even on a
pre-C99 platform.

lint: move lexer code from scan.l to lex.c

Previously, the generated scan.c was not included when running "make
lint". Similarly, cgram.c is still not included.

lint: expand abbreviations in function names

lint: allow system headers to use int as bool, even in strict bool mode

lint: fix return type of conaddr

lint: replace 0 and 1 with false and true, where appropriate

Change in behavior: Passing the option -h exactly 4294967296 times or
any multiple thereof is no longer equivalent to passing it never at all,
it is now equivalent to passing it once. See main2.c, hflag++ for the
actual change.

Other than that, no functional change intended.

A very large portion of the code already conformed to the requirements
of the strict bool mode. The only missing thing was using the constant
literals false and true instead of 0 and 1. For sure there are some
integer literals left that can be converted. For now, all literals that
appeared in the form " = 0" or " = 1" have been replaced.

lint: add new check for strict bool mode

In strict bool mode, bool is considered incompatible with all other
scalar types, just as in Java, C#, Pascal.

The controlling expressions in if statements, while loops, for loops and
the '?:' operator must be of type bool. The logical operators work on
bool instead of int, the bitwise operators accept both integer and bool.
The arithmetic operators don't accept bool.

Since <stdbool.h> implements bool using C preprocessor macros instead of
predefining the identifiers "true" and "false", the integer constants 0
and 1 may be used in all contexts that require a bool expression.
Except from these, no implicit conversion between bool and scalar types
is allowed.

See usr.bin/tests/xlint/lint1/d_c99_bool_strict.c for more details.

The command line option -T has been chosen because all obvious choices
(-b or -B for bool, -s or -S for strict) are already in use. The -T may
stand for "types are checked strictly".

The default behavior of lint doesn't change. The strict bool check is
purely optional.

An example program for strict bool mode is usr.bin/make, which has been
using explicit comparisons such as p != NULL, ch != '\0' or n > 0 in
most places for a long time now, even before the refactoring in 2020.

lint: sprinkle a few const modifiers throughout the code

lint: change return type of eqptrtype to bool

lint: rename functions with short names

lint: rename cvtcon to convert_constant

No functional change.

lint: change return type of typeok to bool

No functional change.

lint: rename ename to enumeration_constant

lint: rename funcarg and funccall to longer names

From the previous short names, it was no obvious that these functions
create a new tree node.

The function named funccall in lint2 has been left as-is, since it has a
completely different prototype.

lint: rename functions that create nodes

lint: rename prflstrg to printflike_argnum, likewise for scflstrg

lint: let gnuism and c99ism return void instead of int

The return value was only used in a single case. Duplicating the
condition for printing a message is ok in that case, since it makes all
other places in the code simpler.

The occasional "(void)" or "msg = " before the function call had hidden
the calls from check-msgs.lua, which didn't check the message texts in
such cases.

lint: convert bitfieldtype_ok from int to bool

No functional change intended, except for the output in debug mode.

lint: split label handling into separate functions

The only thing these cases have in common is the name "label" and the
"reached = 1" assignment. That's not reason enough to combine
completely unrelated functions.

lint: split cluparg into separate functions

That function did more than its short name could express, and the things
it did were not related in any way.

lint: reduce verbosity of assertions

Having 2 lines of source code per assertion is too much, especially
since most of this code is redundant anyway. Extract the common code
and the additional negation into a simple function instead.

lint: split complete_tag into separate functions

lint: rename functions for handling the initialization stack

lint: rename functions with very short names

lint: rename functions that had very short names

C99 guarantees that the first 31 characters of an identifier with
external linkage are significant. This removes the need to use
abbreviations for common words.

lint: rename confusing function setcompl

The previous function name suggested that it would set the complete flag
of the type, but it was the exact opposite. To reduce confusion, negate
the meaning of the parameter.

fix typeof, add __builtin_offsetof

branches: 1.34.2;
teach lint __attribute__((__unused__))

Add -R (source filename remapping) for MKREPRO

branches: 1.32.6;
Handle the rest of gcc __attribute__ s.

branches: 1.31.4;
Allow linted comments to take an argument that defines which error to suppress.

branches: 1.30.4; 1.30.10;
as promised make the last ops table auto-generated.

branches: 1.29.2;
compute sizes by the new tsize() function.

understand __attribute__((__packed__)) and __packed.

Add __alignof__.

Lukemify (WARNS=4)

Don't produce type mismatch warnings if one side of ?: is "[qual] void *"
and the other side is "[qual] pointer".

branches: 1.24.2;
WARNS=4

Add Picky flag; this produces more warnings:
1. long a; int i; a = i * i;
suggests casting i to long, so that we gain precision in the multiplication.
2. warns about magnitude comparisons in enums.
3. warns about possible sign extension issues when integer types become widened.

branches: 1.22.18;
Fix c99 initialization issues. Now the regression tests work.

Factor out tyname() so that it can be used both by lint1 and lint2.
Since type_t is different between lint1.h and lint2.h include the
appropriate file depending on the pass. Make the argument mismatch
error print the type names of the types involved. Now that we have
a tyname() function we can fix the rest of the pass2 warnings to be
more explanatory, but not now.

add c99ism(), by analogy to gnuism(), for c99 construct (un)warnings

add support for ({}) gcc shit.

add C9X/GCC compound literal expressions.

add variable array dimension.

support for c99 style and gnu style structure and union named initializers.

Minimize diffs with my C99 capable version [this commit does not include
C99 support.

- turn lerror() into a macro so that the filename and the line number of the
error are printed before we abort.
- recurse in type printing to provide the proper type name.

Replace u_quad_t with uint64_t and quad_t with int64_t, and use
<inttypes.h> to get those type definitions. These types are more
portable, and a little more sane to do autoconf tests for.

Centralize the initialization/declaration of the ttab.

* Add header files (ilp32.h and lp64.h) that describe the two
models of type sizes that we currently support, and include
the appropriate one in each arch's targparam.h.
* Use the type size constants provided by targparam.h in the
type table, rather than using "sizeof(type) * CHAR_BIT" (which
would get the host's type size, not the target's). XXX Not
yet done for floating point types.
* Add a new BITFIELDTYPE lint comment that suppresses illegal
bitfield type errors if the type is an integer type (e.g.
long, long long), and also suppresses non-portable bitfield
type warnings.

cleanup (prior to more adding more features):
- convert to ANSI KNF
- remove trailing whitespace
- translate some comments from german into english

code compiles and runs clean, and tested by running "make lint" against
xlint source using previous and this lint produces same results.

More format string cleanup by sommerfeld.

new function and new flag...

branches: 1.8.2; 1.8.10;
WARNSify

don't print warnings about unused variables or arguments in compound
statements which contain asm statements.

LINTED and CONSTCOND are now valid up to the next end of a
global or local declaration/definition/statement. Originally they were
valid on the current and next line, which made it hard to suppress
warnings in constructs with more then one line.

LONGLONG can now be used to suppress errors or warnings in the next
declaration, definition or statement.

support asm statements and asm modifiers in declarations

asm statements consist of an asm keyword, an optional qualifier, a
left paren, a list of tokens up to and including the matching right
paren, and a semicolon.

asm modifiers consist of an asm keyword, an left paren, a string and a
right paren.

asm statements and modifiers have no semantic for lint(1), they exist only
to avoid complaints about them.

prototypes override old style function definitions
this is a gnu extension to ansi c

added inline keywords
"inline" is enabled by -g, "__inline" and "__inline__" are always available

RCS id cleanup

branches: 1.1.1;
Initial revision

lint: remove unused flag '-u' from lint1

This flag is completely handled by lint2.

lint: in strict bool mode, allow do-while-0

Even though 0 is not a boolean constant, allow this common idiom, to
help in those cases where the C preprocessor used by lint does not mark
tokens as coming from system headers (Clang).

lint: move code for parsing balanced token sequences to lex.c

To access yytext from lex.c, the lexer needs to be in %pointer mode,
which was specified in IEEE Std 1003.1, 2004 Edition and thus should be
old enough to be available on platforms other than NetBSD, for use in
tools/lint1 where lint1 is built before yacc and lex.

lint: parse but otherwise ignore C23 attributes

The C23 attributes are only parsed before an expression in an expression
statement, as a proof of concept. Other places will follow later.

lint: in C23 mode, support the nullptr constant

lint: measure the alignment in bytes, not bits

While measuring the alignment in bits makes sense when building a struct
type with bit-fields, in all other places it is more confusing than
helpful.

The only visible change is that in debug mode, the format of type sizes
and alignment changed. Since the size of all complete types is a
multiple of a byte (as defined in the C standard), sizes and alignments
are reported in bytes as well. Only while a struct is being built, the
type size may include an additional '+x' for the bits of a bit-field.

lint: support _Alignas and __attribute__((__aligned(4)))

lint: clean up and speed up the check for snprintb

lint: clean up

lint: clean up

lint: use fewer struct keywords

lint: check for unreachable bits and fields in snprintb formats

While here, clean up a few existing checks.

lint: test format strings from snprintb calls

The functions snprintb and snprintb_m are specific to NetBSD, and their
format strings are tricky to get correct. Provide some assistance in
catching the most common mistakes.

lint: make function call arguments directly accessible

Previously, the arguments of a function call expression were arranged in
a linear tree structure, from right to left. To allow easier access to
the arguments, store them in an array instead.

lint: keep strings in their source representation

This allows further analysis depending on whether individual characters are
escaped as octal, hexadecimal or not at all.

lint: clean up comments, reduce scope of variables

lint: use standard buffer for storing string values

No functional change.

lint: rename symt_t to symbol_kind

It was confusing to have two kinds of "symbol type" (s_type and s_symt),
so rename all related identifiers to be more distinctive.

No functional change.

lint: correctly set system-header flag on cast-expression

When a cast-expression comes partly from a system header, determine at
the ')' whether the whole cast-expression comes from the system header.
Previously, it was based on the operand, which contradicted the
documentation of tn_sys.

Mainly affects strict bool mode (where expressions from system headers
are handled more leniently), as well as query 9 for parenthesized return
expressions.

Discovered upon manual inspection, as calling expr_alloc_tnode should
never be necessary when creating an expression node with operands;
there's build_op for that purpose.

lint: allow complex offsetof(type, member-designator)

Both GCC 11 and Clang 8 accept member-designators that are not
identifiers but designator sequences, such as in 'offsetof(struct stat,
st_atim.tv_sec)', so make lint accept them as well.

lint: allow querying for 'static' followed by non-'static' declaration

lint: in declarations, replace tab with space

Previously, in some cases, the keywords 'const' or 'struct' were
followed by a tab, which doesn't make sense.

No functional change.

lint: remove unused string literals

lint: fix handling of unnamed function parameters

lint: distinguish between arguments and parameters

lint: improve debug logging for declarators

lint: in debug mode, default to indenting the debug log

Only the 'parsing' lines are not indented, as line breaks are
independent from the structure of the code.

lint: condense code for ending a function

No functional change.

lint: constify

lint: use standard function attribute for noreturn functions

lint: extract 'vararg' and 'prototype' flags from global 'dcs'

These flags are only relevant for parameter lists, so add a separate
type for it.

No functional change.

lint: clean up duplicate code for finding struct/union members

lint: fix use-after-free bug in GCC statement expressions

lint: add detailed logging for finding memory allocation bugs

lint: use separate types for parsing pointer types and type qualifiers

The documentation of the previous 'qual_ptr_t' was misleading, as that
type was used for other type qualifier lists as well, even if these were
not related to pointer types.

lint: indent copyright lines consistently

lint: reduce code size for handling lint comments

lint: clean up code related to lint comments

Remove type information from variable names, as the word 'flag' did not
indicate that the variables were negated.

Remove contradicting comments. Suppressing a warning for 'this line',
'this and the following line' and 'this statement or declaration' cannot
all be accurate at the same time.

lint: don't treat 'inline' as a storage class

The additional conflict in the grammar is in the same place as the other
conflicts, as T_SCLASS and T_FUNCTION_SPECIFIER are now separate tokens.

No functional change.

lint: warn about function definitions without header declaration

The existing warning was only issued for function declarations, not for
function definitions.

The interesting change in the tests is in msg_351.c. Many other tests
use non-static functions due to their syntactic brevity. In these
tests, the warning is disabled individually, to allow new functions to
be added without generating warning 351.

lint: only skip 'unused' warnings after errors, not other warnings

Previously, in -w mode, any warning suppressed further 'unused'
warnings, even though there was no need to do that. This can be seen in
the test gcc_attribute_var.c, where only the last unused variable from a
function was marked as unused, the others slipped through.

Fixed by counting the errors and the warnings separately and only
combining them if actually desired.

lint: add initial support for C23

Required by xsrc/external/mit/MesaLib.old, brw_eu_validate.c, which
initializes a struct using empty braces: 'return (struct string){};'.

lint: rename 'quad' to 'signed int' or 'unsigned int'

No functional change.

lint: extend debug logging for declaration levels

Indent the debug logging according to the declaration level.

Since there are a few cases where the enclosing declaration levels are
modified, log the whole declaration level stack whenever a level begins
or ends.

lint: clean up duplicate and dead code for integer constants

No functional change.

lint: fix initialization of unnamed union member

lint: clean up names related to declaration levels

The previous prefix 'DK_' (declaration level kind) had a conflict with
the 'DK_' (designator kind) in init.c, so change the prefix to 'DLK_'.
The new name for dinfo_t is decl_level, which is more expressive.

No functional change.

lint: fix handling of unnamed struct/union members

The support for unnamed struct/union members that was added in decl.c
1.60 from 2015-10-13 was simple but wrong. It didn't cover initializers
of these structures and computed wrong sizes for structures containing
anonymous unions. At that time, the handling of initializers was broken
as well, it was fixed 6 years later in init.c 1.229 from 2021-12-22.

Real-life examples for code that lint couldn't handle are:

* external/bsd/jemalloc/dist/src/jemalloc.c
* external/mit/xorg/lib/dri.old/Makefile

lint: clean up handling of declarations

No functional change.

lint: clean up function names

No functional change.

lint: clean up completion of struct, union and enum declarations

No functional change.

lint: remove redundant 'extern' from function declarations

No binary change.

lint: be strict when parsing command line for excluded message IDs

Previously, lint accepted -X '1, 2, 3', while the manual page lists the
IDs without spaces.

On 32-bit platforms, lint accepted -X -4294967295, and on 64-bit
platforms, it accepted the corresponding larger numbers.

The code for parsing message IDs and query IDs conceptually does the
same, but the implementations differed for no reason.

lint: in debug mode, use different names for macro and function

No functional change.

lint: warn about extern declarations outside headers

https://mail-index.netbsd.org/tech-userlevel/2023/03/15/msg013727.html

lint: remove unused internal_error

lint: eliminate unnecessary indirection

No functional change.

lint: rename functions for function definition

No functional change.

lint: remove custom memory allocator

Besides adding complexity, the custom memory allocator didn't invalidate
freed memory, which made it harder to find possible use-after-free bugs.

lint: reduce pointer indirection for unnamed entities

No functional change.

lint: inline previous_declaration for nonconstant message IDs

This adds back the compile-time printf validation in debug mode that was
missing before.

lint: clean up visual clutter

No functional change.

lint: rename functions to be clearer

No need anymore to keep external identifiers at the "6 significant
initial characters" mandated by C90.

lint: rename dcs manipulation functions to be clearer

No functional change.

lint: add additional queries that are not enabled by default

In the last 18 months, several lint warnings have been made adjusted to
allow common usage patterns. For example, lint no longer warns about a
constant condition in the statement 'do { ... } while (false)' (message
161), as this pattern is well-known in statement-like macros, making it
unlikely that the 'false' is a mistake. Another example is casts
between unequal pointer types (message 247) for a few well-known
patterns that are unlikely to be bugs.

Occasionally, it is useful to query the code for patterns or events that
would not justify a warning. These patterns are modeled as predefined
queries that can be selected individually, in addition to and
independently of the existing warnings and errors.

New queries can be added as needed, in the same way as new warnings.
Queries that are deemed no longer used can be deactivated in the same
way as warnings that are no longer used.

As long as none of the queries is enabled, they produce a minimal
overhead of querying a single global variable. Computations that are
more expensive than a few machine instructions should be guarded by
any_query_enabled.

https://mail-index.netbsd.org/source-changes-d/2022/06/28/msg013716.html

ok christos@

lint: add debug logging for struct and enum details

lint: move error handling code from main1.c to err.c

No functional change.

lint: rename mod_t.m_requires_bool to m_compares_with_zero

The operators NOT, LOGAND, LOGOR and QUEST only require _Bool in strict
bool mode, in default mode they accept any scalar expression and compare
it with zero. The new names are more accurate.

No functional change.

lint: unexport lint's messages

There was only a single place where a message was used directly, instead
of calling one of the several functions for this purpose.

lint: inline macro 'sflag'

Mark all places where lint's C90 mode is stricter than its C99 mode.

Most of the situations in which lint produces only warnings instead of
errors covered by the "Constraints" sections in the relevant standards.
This doesn't prevent a specific compiler from accepting it though.

No functional change.

lint: inline macro 'tflag'

The definition of the macro tested both allow_trad and allow_c90, but
there is only a single mode in which allow_c90 is false, therefore it
suffices to test only that.

While double-checking each occurrence of tflag individually, I learned
why lint performs lookups of struct members only by name, independently
of the struct in which they are declared. See typeok_arrow for details.

No functional change.

lint: document why C11 keywords are available in C99 mode as well

lint: inline macro c11flag

No functional change.

lint: migrate gflag to allow_gcc

No functional change.

lint: model C language levels in a future-compatible way

The options -t, -s and -S are confusing because they are used
inconsistently. The option -S enables C99 features, but when using it
instead of -s, it also doesn't enable all checks required by C90 and
later. Prepare fixing of these inconsistencies by replacing the flag
variables with language levels that can be extended in a
straight-forward way as new C standards arrive.

| option | allow_trad | allow_c90 | allow_c99 | allow_c11 |
|--------|------------|-----------|-----------|-----------|
| -t | x | - | - | - |
| (none) | x | x | - | - |
| -s | - | x | - | - |
| -S | - | x | x | - |
| -Ac11 | - | x | x | x |

Each usage of the old flag variables will be inspected and migrated
individually, to clean up the subtle variations in the conditions and to
provide a simpler model.

When lint was created in 1995, its focus was migrating traditional C
code to C90 code. Lint does not help in migrating from C90 to C99 or
from C99 to C11 since there are only few silent changes, and simply
because nobody took the time to implement these migration aids. If
necessary, such migration modes could be added separately.

There is a small functional change: when the option -s is combined with
either -S or -Ac11, lint now only keeps the last of these options.
Previously, these options could be combined, leading to a mixture of
language levels, halfway between C90, C99 and C11. Especially combining
traditional C with C11 doesn't make sense, but xlint currently allows
it.

The 3 tests that accidentally specified multiple language levels have
been adjusted to a single language level.

lint: distinguish between storage class and declaration kind

These types overlap but are not the same.

No functional change.

lint: rename length to length_in_bits

No functional change.

lint: inline setcomplete

No functional change.

lint: add debug logging for the declaration stack

To track down the wrong edge cases in decl_direct_abstract.c.

lint: rename function 'cleanup' to be more expressive

No functional change.

lint: add debug logging for symbols and the symbol table

This logging is not active by default, the functions debug_sym and
debug_symtab can be called as needed during a debug session.

lint: encode lifetime of allocated memory in function names

No functional change.

lint: encode lifetime of allocated memory in the function names

No functional change.

lint: fix memory corruption in statement expressions (since 2021-12-17)

The commit that introduced the assertion failure looks innocent, it only
adds a few predefined functions for GCC mode. Nevertheless, before that
commit, lint consistently complained about 'error: void type illegal in
expression [109]', which doesn't make sense either.

This fix also removes the creative use of the initialization stack to
store the type of the statement expression. Having a separate stack for
these statement expressions makes the code easier to understand.

lint: extract code for handling statement expressions from the grammar

This prepares the fix of the memory corruption bug that is demonstrated
in t_integration.sh, test case assertion_failures.

No functional change.

lint: rename debug_indent to debug_print_indent

The previous name could be mistaken to mean "increase the indentation of
the debug output". Instead, the function prints the current indentation.

In externs1.h, the macro definition was a duplicate, the macros were
sorted differently than the functions a few lines above.

No binary change.

lint: add grammar rule for the beginning of a designation

This will be necessary to properly implement handling of initializers
and braced initializer-lists.

No functional change for now since the designation is already reset
after each expression and '}'. To handle initializations properly, the
designation must not be reset after each expression, it must advance to
the next member instead.

lint: narrow down parameter of build_name

Passing an arbitrary tokenizer symbol left too much freedom and
uncertainty to the caller, and 0 was a magic number in this context.

No functional change.

lint: unexport struct_or_union_member

Strangely, lint2 didn't notice this unnecessary export.

lint: fix check for function calls in strict bool mode

Previously, if a function call occurred in the controlling expression,
its return type could be any scalar, not just bool. This was against
the goal of strict bool mode, which makes bool a separate and
incompabile type to all other types. For example, it would allow
controlling expressions like 'strcmp(a, b)' without the usual '!= 0',
but only if at least one of 'a' and 'b' came from a macro definition
from a system header.

The fix is that the decision of whether the type of the controlling
expression may be scalar is no longer based on the operand types but on
the main operator of the controlling expression.

lint: rename attron to in_gcc_attribute

No functional change.

lint: remove unused function for generic messages

The function message_at is still used, for information about previous
occurrences of a symbol.

lint: add __sync_ and _mm_ as prefixes for builtin functions

These two additions cover all cases that occur in the current NetBSD
build on x86_64. This allows build_name to use the usual pattern 'if
Sflag then error else if sflag then warning'. That function currently
issues a warning in C99 as well, even though C99 prohibits implicit
function declarations.

lint: do not emit GCC builtin functions

Lint1 no longer emits declarations of GCC builtin functions and calls to
them.

Previously, lint generated 3421 useless warnings in a default NetBSD
build, like this:

__atomic_load_n, arg 1 used inconsistently
acl.c(216)[pointer to unsigned int]
rbtdb.c(921)[pointer to unsigned short]

This was because lint just doesn't understand that these functions are
type-generic, which is indeed unusual in C.

These useless warnings made the lint output more frightening than it
should actually be. Together with the strange formatting of the
diagnostics (space-space-tab after the main message, two spaces and two
colons between the occurrences, symbols are listed in hashcode order),
this creates the impression that lint is not intended to be a
user-friendly tool.

For now, fix the excess warnings, leaving the other items for later.

lint: use 'unsigned int' for bit-size of types in convert_integer

There was no need to have two separate magic values (0 and -1) to mean
the same.

No functional change.

lint: use 'unsigned int' for bit-size of types

Lint does not need to support any types larger than 256 MB since they
don't occur in practice. Practically, such large types have never been
supported at all since the function type_size_in_bits used int for the
internal calculations, resulting in overflows.

lint: fix a few lint warnings about type conversions

A simple 'unsigned int' is more than enough for representing the size of
a bit-field, as well as the maximum alignment of any type.

No functional change.

lint: inline msb

This reduces the binary size on x86_64 by about 500 bytes.

No functional change.

lint: remove unnecessary width parameter for msb

No functional change.

lint: remove Tflag and pflag from common declarations

These flags are only needed by lint1 and lint2, but not by the driver
xlint.

No functional change.

lint: change return type of 'msb' from int to bool

No functional change.

lint: merge duplicate debugging code

The functions 'debug_node' and 'display_expression' were similar enough
to be merged.

Migrate debug_node to use the existing debug logging functions.

Remove the now unused option 'd' from the options string.

lint: remove option -d, clean up debug logging

The command line option -d was not used by /usr/bin/lint, and it only
triggered a handful of debug messages. Move this debug logging over to
the compile-time -DDEBUG setting.

Move display_expression further up to avoid the forward declaration.

lint: do not evaluate arguments of debug_step

To analyze the unexpected test failure of op_shl_lp64, I had reverted
debug_step to evaluate its arguments. I then accidentally committed
that without running the tests again.

Anyway, the previous commit can now be used as a demonstration that
initdecl is indeed missing the initialization for __uint128_t, which
leads to the internal error in op_shl_lp64.

lint: clean up debug logging

The calls to debug_step, unlike printf, don't need a trailing newline.

Remove the debug_step0 macro and its relatives since lint already uses
enough other features from C99 that it essentially requires this
standard, which supports varargs macro arguments. Among these features
are __func__ and printf("%zu").

In non-debug mode, do not evaluate the arguments of debug_step.
Evaluating the arguments had caused an internal error when running the
test op_shl_lp64. This is indeed a bug since initdecl should have
initialized the type table for __uint128_t. This had been forgotten
when support for __uint128_t was added in decl.c 1.69 from 2018-09-07.

No functional change.

lint: extract debug logging to separate file

Lint currently has several different kinds of debug log:

* The -DDEBUG log is controlled at compile time.
* The -d command line options enables some other debug logging.
* The -DYYDEBUG log for parsing is controlled at compile time.
* The -y command line option only has an effect in -DYYDEBUG mode.

Extracting the logging into a separate file is a first step towards
unifying these logs and making the code for debug logging stand out less
than the current #ifdef DEBUG.

No functional change.

lint: add debugging output for the grammar tokens

No functional change outside debug mode.

lint: merge duplicate code for generating unqualified type

This is a preparation for fixing the wrong warnings in msg_115.c.

No functional change.

lint: move scl_name from print.c to decl.c

It is only needed there.

lint: use consistent naming scheme for functions that build nodes

No functional change.

lint: split 'build' into build_binary and build_unary

No functional change.

lint: replace call to ttos with type_name

Since tyname.c 1.20 from 2021-01-02, type_name is as simple to use as
ttos and more expressive. It can also be called multiple times without
invalidating the returned strings.

Used only in debug mode.

lint: extract build_member_access from the grammar

No functional change.

lint: rename clrtyp/deftyp to begin_type/end_type

The abbreviations clr/def did not make it obvious that these two
functions or grammar rules form pairs.

No functional change.

lint: rename type generic_association_types to generic_association

The word 'types' was misleading and unnecessary.

No functional change.

lint: rename xsign to convert_integer

The term sign-extend was too specific, the function actually does a
broader conversion.

No functional change.

lint: rename pqinf_t to qual_ptr and clean up code

The 'inf' from the type name meant 'information' and was redundant. Each
object of that type represents a single pointer level, which made the
documentation about 'pointers' a bit confusing.

The members of struct qual_ptr are now in the canonical reading order,
which is 'const volatile pointer'.

No functional change.

lint: fix result type of _Generic expressions

lint: remove unused print_tnode

The function display_expression serves the same purpose and prints more
details.

lint: in debug mode, print name of the "storage class"

This helps in tracking down the internal errors related to declarations,
found by running afl.

lint: replace array access with function calls

First and foremost, the test d_c99_complex_split accessed the array
qlmasks out-of-bounds, with an index of 128 for the type 'double
_Complex'. This invoked undefined behavior since the maximum allowed
index was 64.

Replacing the raw array accesses with function calls allows for bounds
checks to catch these errors early.

Determining the value bits for a 'double _Complex' does not make sense
at all since it is not an integer type. This means that lint didn't
handle these types correctly for several years. Support for int128_t
has been added in inittyp.c 1.12 from 2018-09-07, support for _Complex
has been added in inittyp.c 1.9 from 2008-04-26.

Determining the value bits for an int128_t would make sense, but the
unit tests don't contain examples for this type since at the moment all
unit tests must produce the same results on 32-bit and 64-bit platforms,
and the 32-bit platforms don't support int128_t.

lint: pass pos_t via const pointer

Thanks for the suggestion, christos@.

lint: add error_at, warning_at, message_at

Right now there are several places in the code that use the global
variable curr_pos for passing a parameter to the diagnostic functions.
That's not what global variables are for.

Make it easy for the code to migrate to the parameter-passing style.

No functional change.

lint: do not allow the diagnostics to be modified

lint: add option to accept C11 features

The list of available letters for the command line options gets shorter
and shorter. Most of the interesting letters are already used for some
warning categories. Curiously, -A, -W and -E were all still available.

The option -A nicely matches the intention of the option, which is to
allow a certain set of language features. To keep the option available
for further extensions, define -Ac11 as the currently only valid option
of that kind. This allows straight-forward extension for C17 and future
language standards, as well as independent feature-sets. The options -W
and -E may someday complement the -A option, using the allow/warn/error
categories.

lint: add support for C11-isms such as int[static 3]

lint: clean up handling of preprocessing output lines

No functional change.

lint: in code from included files, print stack trace

Previously, the standard NetBSD build generated several lint warnings in
lhash.h from OpenSSL, without providing any hint as to which file
actually included that header. In cases like these, lint now interprets
the line number information in the preprocessor output from GCC to
reconstruct the exact include path to the file in question.

The program check-expect.lua had to be rewritten almost completely since
it assumed that all diagnostics would come from the main file. In all
existing tests, this was true, but these tests did not cover all cases
that occurred in practice. Now it records the complete location of the
diagnostic instead of just the line number.

lint: move check for strict bool mode into separate file

No functional change.

lint: warn about for wrong type cast in argument to ctype.h functions

The argument to most of the functions from <ctype.h> "shall either be
representable as an 'unsigned char' or shall equal the value of the
macro EOF".

When confronted with the infamous warning 'array subscript has type
char', there are enough programmers who don't know the background of
that warning and thus fix it in a wrong way. Neither GCC nor Clang
explain its warning to target these programmers.

Both GCC and Clang warn about 'array subscript has type char', but they
ignore the other requirements of the <ctype.h> functions, even though
these are in the C standard library.

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94182
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95177
https://stackoverflow.com/a/60696378

lint: rename functions for duplicating types

No functional change.

lint: name memory allocation functions consistently

No functional change.

lint: rename getnode to expr_zalloc_tnode

The new name highlights that the returned memory is only valid in the
scope of the current expression. This was misleading before since the
other related functions all have a 't' (probably for 'temporary') in
their names.

Also encode in the function name that the returned memory is zeroed out
as that could not be inferred from the old name.

No functional change.

lint: rename mbl to memory_block

No functional change.

lint: rewrite handling of initializations, fixing several bugs

The previous implementation had a wrong model of how initialization
happens in C99, its assertions failed in all kind of edge cases and it
was not possible to fix the remaining bugs one at a time without running
into even more obscure assertion failures.

The debug logging was detailed but did not help to clarify the
situation. After about 20 failed attempts at fixing the small details I
decided to start all over and rewrite the initialization code from
scratch. I left the low-level parts of handling designators, the code
that is independent of brace_level and the high-level parts of how the
parser calls into this module. Everything else is completely new.

The concept of a brace level stays since that is how C99 describes
initialization. The previous code could not handle multi-level
designations (see d_init_pop_member.c). There are no more assertion
failures in the initialization code.

Some TODO comments have been left in the tests to keep the line numbers
the same in this commit. These will be cleaned up in a follow-up
commit.

The new implementation does not handle initialization with "missing"
braces. This is an edge case that both GCC and Clang warn about, so it
is not widely used. If necessary, it may be added later.

The new implementation does not use any global variables in the vast
majority of the functions, to make all dependencies and possible
modifications obvious.

lint: sprinkle const on function declarations

No functional change.

lint: rename public designator functions

Their previous names were too similar to the actual implementation
functions, which was confusing.

No functional change.

lint: rename decl1ext, reduce visibility

No functional change.

lint: rename LERROR to INTERNAL_ERROR

The '#ifndef' in tyname.c is meant to distinguish between lint1 and
lint2, it is not meant to be defined from anywhere outside the lint code
itself.

No functional change.

lint: rename filename management functions

No functional change.

lint: rename fnaddreplsrcdir to something less cryptic

No functional change.

lint: remove unused fnalloc

lint: rename pushctrl and popctrl to be more expressive

While here, remove the magic number 0 that in this context means
"function body".

No functional change.

lint: rename pushdecl and popdecl to be more expressive

The previous names were highly ambiguous. The 'decl' could have meant
'declaration', which would be the usual abbreviation. It could also be
split into 'dec' and 'l', meaning 'declaration level', which would make
more sense in this particular context.

To avoid having to guess anything about these names, rename the
functions. Instead of 'push' and 'pop', I renamed them to 'begin' and
'end' since these are the high-level operation that are of interest.
That the hierarchy of declaration levels is implemented as a stack is
nice to know but not as important to understand the whole situation.

No functional change.

lint: fix names of functions dealing with designations

These functions modify the whole designation, not only a single
designator.

No functional change.

lint: fix wrong error message about type mismatch in compound literal

Now that the code contains explicit markers for starting and ending an
initialization, and having the guarantee that an assertion fails
whenever some code accesses the state of the "current initialization"
even though there is no ongoing initialization gives me much more
confidence in the correctness of the code. The calls to
begin_initialization and end_initialization always appear in pairs,
enclosing the minimal amount of code necessary for initialization.

In a nutshell, global modifiable state is error-prone and hard to
understand.

A nice side effect is that the grammar no longer needs a special rule
for the outermost initializer since the functions for the debug logging
are now called explicitly.

The code that misuses the initialization state just because it needs to
temporarily store a sym_t somewhere is now clearly marked as such. A
GCC statement expression can appear anywhere and is therefore
independent of the initialization. Most probably the code can simply
refer to the local variable in the grammar rule itself, or this variable
needs to be encoded in the grammar %union. For sure there is a better
way to handle this.

There is no longer a need that the function 'declare' initializes the
initialization state, it was just the wrong place to do this.

lint: add indirection for accessing the current initialization

This indirection will be needed to handle nested initializations, which
are a new feature of C99. These are currently not handled correctly,
see msg_171.c.

No functional change.

lint: invert 'rchflag', call it warn_about_unreachable instead

No functional change.

lint: rename functions for handling control statements

No functional change.

lint: rename ftflg to seen_fallthrough

One less abbreviation to remember when reading the code.

No functional change.

lint: use standard form of statement macro for debug_node

No functional change.

lint: rename tsize to type_size_in_bits

The shorter name size_in_bits was already taken by the function-like
macro with argument type tspec_t.

No functional change.

lint: rename push_member and pop_member

These two functions are supposed to model the designator that is used
for initializing structs and arrays. The implementation is still buggy
and does not work at all for C99 designators with multiple names, see
d_init_pop_member.c.

For now, just rename the functions to head in the right direction.

No functional change.

lint: add debug logging for C99-style initialization of arrays

No functional change outside debug mode.

lint: move to_int_constant from the grammar to decl.c

This way, the code is covered by running 'make lint'. The code from the
grammar is not covered, therefore it still uses int instead of bool in a
few places.

Inline the comparison functions for uint64_t. These functions didn't
add any clarity to the code.

No functional change.

lint: move main part of idecl over to decl.c and rename it

No functional change.

lint: rename 'blklev' to 'block_level'

No functional change.

lint: rename getbound to alignment_in_bits

No functional change.

lint: extract check_non_constant_initializer from init_using_expr

No functional change.

lint: fix definition of debug_node in non-debug mode

lint: indent node details in debug mode

lint: extend debugging for initializing objects

No functional change outside debug mode.

lint: rename mkinit to init_using_expr

No functional change outside debug mode.

lint: warn about mismatch in getopt handling

lint: don't warn about constant condition in 'do { } while (0)'

lint: rename incompl to is_incomplete

No functional change.

lint: expand abbreviations in lexer function names

No functional change.

lint: apply strict bool mode to lex.c

There are 2 remaining expressions:

In line 244, !(kw->kw_deco & deco) is a bitwise and. This is already
allowed for enums, it needs to be allowed for arbitrary integer
expressions as well. This covers the many places where plain integers
are used for bit fields, together with #define. This pattern is not as
typesafe as using enums, still it is common practice.

In line 769, the expression !finite(f) is a legitimate use of a function
that has return type int for traditional reasons. It's the same as for
ferror.

There are several other functions like unlink, open or strcmp that have
return type int as well, but with a different meaning. It is not yet
clear what the best way is to handle these different meanings. Having
to write finite(f) == 0 in strict bool mode doesn't look idiomatic, on
the other hand, !strcmp(s1, s2) is exactly the pattern that strict bool
mode wants to avoid.

lint: fix two wrong error messages in strict bool mode

The strict bool mode gets complicated because for system headers the
rules need to be relaxed since they cannot be changed easily, often not at all.

Still, if lint validates a program in strict bool mode, that program
must run with equal behavior regarding boolean expressions even on a
pre-C99 platform.

lint: move lexer code from scan.l to lex.c

Previously, the generated scan.c was not included when running "make
lint". Similarly, cgram.c is still not included.

lint: expand abbreviations in function names

lint: allow system headers to use int as bool, even in strict bool mode

lint: fix return type of conaddr

lint: replace 0 and 1 with false and true, where appropriate

Change in behavior: Passing the option -h exactly 4294967296 times or
any multiple thereof is no longer equivalent to passing it never at all,
it is now equivalent to passing it once. See main2.c, hflag++ for the
actual change.

Other than that, no functional change intended.

A very large portion of the code already conformed to the requirements
of the strict bool mode. The only missing thing was using the constant
literals false and true instead of 0 and 1. For sure there are some
integer literals left that can be converted. For now, all literals that
appeared in the form " = 0" or " = 1" have been replaced.

lint: add new check for strict bool mode

In strict bool mode, bool is considered incompatible with all other
scalar types, just as in Java, C#, Pascal.

The controlling expressions in if statements, while loops, for loops and
the '?:' operator must be of type bool. The logical operators work on
bool instead of int, the bitwise operators accept both integer and bool.
The arithmetic operators don't accept bool.

Since <stdbool.h> implements bool using C preprocessor macros instead of
predefining the identifiers "true" and "false", the integer constants 0
and 1 may be used in all contexts that require a bool expression.
Except from these, no implicit conversion between bool and scalar types
is allowed.

See usr.bin/tests/xlint/lint1/d_c99_bool_strict.c for more details.

The command line option -T has been chosen because all obvious choices
(-b or -B for bool, -s or -S for strict) are already in use. The -T may
stand for "types are checked strictly".

The default behavior of lint doesn't change. The strict bool check is
purely optional.

An example program for strict bool mode is usr.bin/make, which has been
using explicit comparisons such as p != NULL, ch != '\0' or n > 0 in
most places for a long time now, even before the refactoring in 2020.

lint: sprinkle a few const modifiers throughout the code

lint: change return type of eqptrtype to bool

lint: rename functions with short names

lint: rename cvtcon to convert_constant

No functional change.