make: sync VarEvalMode constant names with their debug log names

make: fix a few more memory leaks

make: clean up comments, code and tests

make: remove redundant comments

No functional change.

make: refactor CondParser_Term to be inlinable

No functional change.

make: clean up parsing of conditions

No functional change.

make: clean up comments

No binary change, except for line numbers in assertions.

make: simplify memory allocation for string buffers

In edge cases and short-lived buffers, the initial buffer size is
irrelevant, so use the default.

No functional change.

make: clean up comments

No binary change, except for line numbers in assertions.

make: clean up names of local variables

No binary change.

make: replace 'variable expression' with 'expression' in comments

No binary change.

make: clean up multiple-inclusion guards

No functional change.

make: clean up variable and function names

No functional change.

make: warn about malformed patterns in ':M', ':N' and '.if make(...)'

These patterns shouldn't occur in practice, as their results are tricky
to predict. Generate a warning for now, and maybe an error later.

Reviewed by sjg@.

Allow guard targets to use variables.

I commonly use __${.PARSEDIR:tA}__ where a unique guard
is needed, __${.PARSEDIR}__ is also useful in many cases.

Combination of patch from rillig and mine

make: allow targets to be used as multiple-inclusion guards

This style is used by FreeBSD, among others.

make: add tests for full code coverage of multiple-inclusion guards

make: clean up code for skipping files with multiple-inclusion guard

No functional change.

make: if a makefile is protected by a guard, only include it once

"looks reasonable" sjg@

make: remove parameter names from function declarations

No binary change.

make: shorten function names, clean up comments

No functional change.

make: reduce complexity of evaluating expressions

No functional change.

make: don't interpret the return value of Var_Parse

The return value of Var_Parse is largely redundant to the returned
string. The idea behind the type VarParseResult was to migrate all call
sites to checking this return value instead of the returned string, but
that hasn't happened. Instead, the additional type only added more
complexity.

There was a single place where that return value was actually used, when
parsing conditions. And even in that case, ignoring the VarParseResult
added back an error message that previously hid bugs, in the test
cond-token-plain.mk.

Even though these error messages are redundant in the other tests, they
don't hurt as they don't happen often.

make: clean up tracking of depth of nested .if directives

The variable cond_min_depth was redundant. It was only accessed while
parsing the makefiles. Merging it into struct IncludedFile removes the
possible confusion between cond_min_depth and including_cond_min_depth.

No functional change.

make: fix variable and function names relating to .if nesting

The previous names were confusing since they suggested that cond_depth
instead of cond_min_depth would be saved and restored.

No functional change.

make: move Cond_save_depth above Cond_restore_depth

This puts the functions into chronological order, as saving happens
before restoring.

No functional change.

make: use assertion for internal error condition

When the nesting level of conditionals is restored to an unreasonably
high value, the error message "0 open conditionals" doesn't make sense.

Cond_reset_depth just use cond_min_depth

To avoid errors from unclosed conditionals on .break
it is sufficient to just set cond_depth = cond_min_depth.

Patch from rillig

make: list comparison operators in declaration order

This allows a tiny optimization in the switch statement.

No functional change.

make: add more details to error message for numeric comparison

Before:
String comparison operator must be either == or !=

After:
Comparison with '>=' requires both operands 'no' and '10' to be numeric

Noticed by martin@ in pkgsrc/textproc/py-pygments.

make: add .break to terminate .for loop early

When .break is encountered within a .for loop
it causes immediate termination.

Outside of a .for loop .break causes a parse error.

Reviewed by: christos

tests/make: adjust expectations to actual behavior

The bug in deptgt-silent-jobs.mk has been fixed, the debug logging for
comparing conditions and for deleting global variables has changed
intentionally.

make: improve comments about parsing and evaluating conditions

No binary change.

make: improve local variable name in ParseWord in conditions

That function not only parses function arguments but also bare words, so
the name argBuf didn't match anymore.

No binary change.

make: make debug logging for comparisons less technical

make: simplify control flow in CondParser_Comparison

No functional change.

make: fix mistakes, spelling and typos in comments and manual page

No binary change for -DNDEBUG.

make: clean up variable names

No binary change.

make: rename labels in CondParser_Leaf

No binary change.

make: replace Var_Value with Var_Exists where applicable

The latter function already existed in 1993, no idea why it was not
used.

No functional change.

make: inline EvalBare

No functional change.

make: clean up function argument names and comments

No binary change except for assertion line numbers.

make: remove redundant initializer in CondParser_ComparisonOrLeaf

No binary change.

make: clean up nitpicks

In ParseWord, the expressions '*p' and 'ch' are the same.

In ParseDependencyTargetWord, clean up a wordy comment and join two
conditions.

In the test cond-token-number, clarify that make doesn't convert from
hex to decimal but only from hex to internal representation.

No functional change.

tests/make: demonstrate edge case that evaluates an expression twice

make: remove redundant code from CondParser_ComparisonOrLeaf

No functional change.

make: split ParseWord into the actual ParseWord and ParseFuncArg

Combining two similar but fundamentally different parsing tasks in a
single function only increased the complexity, of the implementation as
well as the call sites.

The code makes it obvious now that a function argument is a bare word
surrounded by parentheses.

The special case of an empty word is only needed for the function
argument, it cannot occur in a bare word. The code for that has been
moved to the caller. Such an empty word not only occurs for 'defined()'
but also for 'defined(${:U})'.

No functional change.

make: make ParseWord in condition parser simpler

Merge the two return values (bool, string) into a single return value.
As before, the caller cannot observe the difference between a parse
error and an empty word, both are handled in the same way.

In CondParser_ComparisonOrLeaf, the word cannot be empty since the
calling function CondParser_Token already handles all cases that could
lead to an empty word.

No functional change.

make: internally return false for irrelevant leaves in conditions

The result of irrelevant leaves is effectively ignored by CondParser_And
and CondParser_Or. Use the 'doEval &&' pattern to make the code
consistent with CondParser_Comparison and CondParser_FuncCall.

No functional change.

make: in irrelevant function calls in conditions, return false

When a condition contains an irrelevant function call, it doesn't matter
whether the function call evaluates to true or to false, it will be
discarded anyway by either CondParser_And or CondParser_Or.

Returning false instead of true makes the code simpler, plus it is more
common to return false for irrelevant results.

No functional change.

make: use simpler return type for ParseWord in conditions

No functional change.

make: replace table for function lookup in conditions with simple code

The code for looking up the function from the table forced the compiler
to use a specific memory layout. Replacing the table with explicit code
provides the compiler more opportunities to optimize the code. Another
side effect is that there are fewer pointer operations.

Previously, is_token checked that the character after the word does not
continue the word, this is now done separately since for the function
lookup, this check was unnecessary. The newly added skip_string
provides a higher abstraction level, it is no longer necessary to pass
the string length as a separate, redundant parameter.

No functional change.

make: clean up condition parser

No functional change.

make: merge duplicate types CondEvalResult and CondResult

No binary change.

make: merge duplicate enum constants for CondEvalResult and CondResult

No binary change.

make: remove redundant parameter for evaluating conditions

No functional change.

make: merge types CondResult and CondEvalResult

No functional change.

make: rename local variables to be simpler

No binary change.

make: clean up comments

make: format comments according to /usr/share/misc/style

Assisted by indent(1), with manual corrections due to its many remaining
bugs.

No functional change.

make: use consistent indentation for statements and continuations

No binary change, except for line numbers in assertions in suff.c.

make: condense CondParser_ComparisonOp

No functional change.

make: fix memory leak when parsing '.if ${expr}' (since 2004-04-13)

$ (
echo 'VAR=value'
perl -e 'printf(".if \${VAR}\n.endif\n" x 5000);';
echo 'all:'
printf '\t%s\n' \
'@pid=$$$$; \' \
'ppid=$$(ps -o ppid -p "$$pid" | sed 1d); \' \
'ps -o vsz,rsz -p $$ppid | sed 1d'
) > leak-cond2.mk

$ make-2004.04.08.07.24.26 -r -f leak-cond.mk
2668
$ make-2004.04.13.16.06.23 -r -f leak-cond.mk
3964

$ echo $(((3964 - 2668) * 1000 / 5000))
259

This 259 is close enough to the 256 from Buf_Init in CondParser_Leaf.

tests/make: refine comments for parsing and evaluating conditions

make: rename ParseFuncArg to ParseWord, update comments

No functional change.

make: merge duplicate code in CondParser_FuncCall

No functional change.

make: generate more specific error message on malformed conditional

make: remove dead code in CondParser_FuncCall

The return value (size_t)-1 was only used by the function 'empty', which
moved to CondParser_FuncCallEmpty.

No functional change.

make: remove unused function parameter in conditional handling

No functional change.

make: clean up CondParser_FuncCallEmpty

No functional change.

make: inline ParseEmptyArg into CondParser_FuncCallEmpty

No functional change.

make: simplify evaluation of function 'empty'

No functional change.

make: merge duplicate code in ParseEmptyArg

No functional change.

make: use proper return type for ParseEmptyArg

Now that ParseEmptyArg is no longer bound to have the same prototype as
ParseFuncArg, it can use a more appropriate return type instead of
encoding everything in a size_t.

Since ParseEmptyArg never returned 0, that code path was unused. It was
only used for all other functions.

No functional change.

make: remove unused arguments from CondParser_FuncCallEmpty

No functional change.

make: separate handling of 'empty' and the other functions

No functional change.

make: merge duplicate code in parsing conditions

No functional change.

make: remove recursion from CondParser_And

No functional change intended. Before cond.c 1.286 from today, there
would have been the functional change that in malformed conditions, the
extra expression would not be evaluated. Now that CondParser_Token is
always called with doEval == false, there is no change in behavior to be
expected.

make: do not expand operator token in CondParser_Or

At the point where CondParser_Or calls CondParser_Token, there was a
previous call to CondParser_And. Due to this, the next token is already
stored in par->curr, and the parameter doEval is ignored.

Changing the argument from doEval to false makes the code similar to the
corresponding code in CondParser_And.

No functional change.

make: do not evaluate misplaced expressions in malformed conditions

This change only affects the behavior for parse errors. Syntactically
well-formed conditions work exactly as before.

make: simplify parsing of '||' in conditions

Previously, the grammar said 'Or -> Or || And', while the code looked
more like 'Or -> And || Or'. Make the code look like the grammar and
keep track of the resulting value of the condition explicitly.

No functional change intended.

make: revert previous commit to CondParser_Or

Even though the unit tests for make cover a lot of cases, they don't
cover all cases. After the previous commit, the NetBSD build failed
with:

bsd.sys.mk line 247: \
Malformed conditional (!defined(NOPIE) && \
(!defined(LDSTATIC) || ${LDSTATIC} != "-static"))

make: avoid recursion in CondParser_Or

Previously, a long chain of '1 || 1 || 1 || 1 || ...' led to a deep
recursion. Furhermore, the code didn't match the grammar on superficial
reading: the grammar said "or || and", the code said "and || or".

No functional change.

make: remove period from end of error messages and warnings

The majority of the existing error messages and warnings does not
include a period at the end. Follow this style consistently.

make: fix comments

make: fix lint warning on i386

cond.c(800): warning: argument #3 is converted from 'unsigned char' to
'unsigned int' due to prototype [259]

make: reduce relocations, thereby reducing .text size

No functional change.

make: reduce relocations and thereby .text size, make opname const

No functional change.

make: rename and invert CondParser.lhsStrict to leftUnquotedOK

The new name is easier to grasp than the rather abstract word 'strict'.

No functional change.

make: do not allow unquoted 'left == right' after modifier ':?'

Having a static variable for state that clearly belongs in the parser
looked suspicious, and indeed it was wrong.

When the distinction between .if conditions and expressions of the form
${condition:?:} was added in cond.c 1.68 from 2015-05-05, a new unit
test was added, but it didn't cover this edge case. At that time, the
state of the condition parser consisted of a few global variables
instead of a separate data type, as would have been appropriate for
parsing nested conditions.

make: remove unnecessary negation from CondParser_StringExpr

No functional change.

make: remove unnecessary const from parameters

These were leftovers from earlier refactorings, when extracting code to
separate functions.

No functional change.

make: fix out-of-bounds memory read (since previous commit)

make: reduce indentation in CondParser_FuncCall

No functional change.

make: fix documentation about operator associativity in conditions

make: in TryParseNumber, reset errno as late as possible

No functional change.

make: fix grammar in error message for malformed conditional

make: clean up comments in cond.c

make: inline strcmp when parsing conditions

GCC 10 does not do that even though it could easily.

No functional change.

make: clean up function names and documentation for conditions

No functional change.

make: rename If_Eval to EvalBare

No functional change.

make: move parsing of function calls out of CondParser_LeafToken

The grammar above the parsing code says that a Leaf has nothing to do
with function calls, so don't mix these in the actual code.

No functional change.

make: improve function names for parsing conditions

No functional change.

make: do not complain when skipping the condition 'no >= 10'

Seen in external/bsd/tmux when building with Clang. See
varmod-ifelse.mk for the detailed story.

make: convert VarEvalFlags back into an enum, but not a bit-set

As was apparent in VarEvalFlags_ToString, a bit-set was not the best
data type since most of the flags were not freely combinable. The two
flags that could be combined were keepDollar and keepUndef, but even
these have distinguished names in the debug log.

The downside of struct bit-fields is that they need extra helper
functions in C90 (see nonints.h). Exchange these for a few helper
functions in var.c, to keep the code outside var.c simple.

No functional change.

make: use C99 bool type instead of defining its own

No functional change.

make: replace enum bit-field with struct bit-field for VarEvalFlags

This makes the code easier to read, especially in var.c. It also makes
debugging sessions easier since some debuggers don't show enum
bit-fields symbolically as soon as more than one bit is set.

The code outside var.c is basically unchanged, except that instead of
passing the individual flags, there are 4 predefined evaluation modes.
These suffice for all practical use cases. Only in the implementation
deep inside var.c, the value of the flags keepDollar and keepUndef
differs.

There is no way of passing the struct to EnumFlags_ToString, which means
the ToString function has to be spelled out explicitly. This allows for
fine-tuning the representation in the debug log, to reduce the amount of
uppercae letters.

No functional change.

make: rename VARE_NONE to VARE_PARSE_ONLY

The name 'NONE' described the bit pattern, which was not useful to
understand its meaning. Omitting VARE_WANTRES only parses the
expression, without evaluating any part of it.

No functional change, not even in debug mode since Enum_FlagsToString
always returns "none" for all-bits-unset.

make: remove freestanding freeIt variables

These variables all belong to a string variable. Connect them using
FStr, which reduces the number of variables to keep track of.

No functional change.

make: in the Var_ functions, move the scope to the front

This change provides for a more natural reading order in the code.
Placing the scope first makes it immediately clear in which context the
remaining parameters are interpreted.

No functional change.

make: rename some VAR constants to SCOPE

The word "context" does not fit perfectly to the variables that are
associate with a GNode, as the context is usually something from the
outside and the variables are more like properties inherent to the
GNode.

The term "global context" fits even less. Since the thing where
variables are looked up is commonly named a scope, use that term
instead.

This commit only renames the global variables VAR_GLOBAL, VAR_INTERNAL
and VAR_CMDLINE, plus a few very closely related comments. These are:

GNode.vars (because of line breaks)
GNode_Free (dito)
varname-make_print_var_on_error.mk
varname-make_print_var_on_error-jobs.mk

The debug message in Var_Stats is left as-is since there is no unit test
for it yet.

The other renamings (variable names "context", "ctxt", as well as
further comments) will be done in a follow-up commit.

make(1): split Buf_Destroy into Buf_Done and Buf_DoneData

In all cases except one, the boolean argument to Buf_Destroy was
constant. Removing that argument by splitting the function into two
separate functions makes the intention clearer on the call site. It
also removes the possibility for using the return value of Buf_Done,
which would have made no sense.

The function Buf_Done now pairs with Buf_Init, just as in HashTable and
Lst.

Even though Buf_Done is essentially a no-op, it is kept as a function,
both for symmetry with Buf_Init and for clearing the Buffer members
after use (this will be done only in CLEANUP mode, in a follow-up
commit).

make(1): remove "warning" from missing closing parenthesis

This only affects the diagnostics for parse errors that involve a
missing closing parenthesis. Whether or not this is a parse error is
still the same.

It may look redundant to pass both the CondParser and the parsing
position pp to the functions, but that's necessary since during parsing,
not every code path updates the main parsing position immediately.

make(1): replace warning + error with just an error in conditionals

Before, there was a "warning" for comparing strings using '<', which was
wrong. That warning was then followed by an error, after parsing the
whole conditional. This was only because it was easier to implement.

Replace the warning with an actual error. This only affects
conditionals in .if lines, the conditionals in the :? modifier such as
${"A" < "B":?smaller:greater} still print 2 errors.

make(1): fix debug output for comparison operators in conditionals

This produces fewer warnings than before, but these were edge cases that
probably didn't matter in practice. The "Malformaed conditional" is
still generated, the set of accepted conditionals is still the same.

make(1): document wrong debug output in evaluation of conditionals

make(1): reformat CondParser_StringExpr

make(1): extract CondParser_StringExpr from CondParser_String

make(1): demonstrate parse error without error message in conditional

make(1): make CondParser_String simpler

No functional change.

make(1): fix Coverity annotation

FStr is not an allocated data type.

make(1): extract DetermineKindOfConditional from Cond_EvalLine

make(1): remove struct If from evaluation of conditionals

The variant of the .if directive can be stored in the CondParser
directly, instead of pointing to a struct If.

The type CondParser may have to be renamed since the condition is not
only parsed, it is evaluated as well.

No functional change.

make(1): remove do-not-format markers from comments

These markers had been used inconsistently. Furthermore the source code
had not been formatted automatically before 2020 at all, otherwise there
wouldn't have been any trailing whitespace left.

make(1): distinguish between tokens and results in conditionals

No functional change.

make(1): update references to the renamed CondParser functions

make(1): condense code for parsing and evaluating conditionals

make(1): add error handling for edge case in malformed conditions

make(1): fix possible return values for CondParser_Term

The invalid return values didn't do any harm since CondParser_Factor and
CondParser_Expr passed them through, and CondParser_Eval carefully
checks for TOK_TRUE or TOK_FALSE and treats everything else as an error.

No change in observable behavior since there is no debug logging in that
part of the code.

make(1): demonstrate wrong return value in CondParser_Term

make(1): consistently use boolean expressions in conditions

Most of the make code already followed the style of explicitly writing
(ptr != NULL) instead of the shorter (ptr) in conditions.

The remaining 50 instances have been found by an experimental,
unpublished check in lint(1) that treats bool expressions as
incompatible to any other scalar type, just as in Java, C#, Pascal and
several other languages.

The only unsafe operation on Boolean that is left over is (flags &
FLAG), for an enum implementing a bit set. If Boolean is an ordinary
integer type (the default), some high bits may get lost. But if Boolean
is the same as _Bool (by compiling with -DUSE_C99_BOOLEAN), C99 6.3.1.2
defines that a conversion from any scalar to the type _Bool acts as a
comparison to 0, which cannot lose any bits.

make(1): fix lint warnings

make(1): format multi-line comments

make(1): clean up VarParseResult constants

The many constants were invented because at that time I didn't quite
understand the actual outcomes of Var_Parse that need to be
distinguished. There are only a few:

(1) Errors, whether they are parse errors, or evaluation errors or
undefined variables. The old constants VPR_PARSE_MSG and
VPR_UNDEF_MSG are merged into VPR_ERR.

(2) Undefined expressions in a situation in which they are allowed.
Previously the documentation for VPR_UNDEF_SILENT talked about
undefined expressions in situations where they were not allowed.
That case is fully covered by VPR_ERR instead.

(3) Errors that are silently ignored. These are probably bugs.

(4) Everything went fine, the expression has a defined value.

make(1): rename CmdOpts.lint to strict

When running lint(1) on the code, it defines the preprocessor macro
"lint" to 1, which generated a syntax error in the declaration "Boolean
lint", as that became "Boolean 1".

make(1): change return type of Var_Value to FStr

make(1): return FStr from Var_Parse

This reduces the number of variable declarations at the call sites.

make(1): clean up memory handling in CondParser_String

make(1): clean up memory handling in CondParser_String

make(1): error out on .else with extraneous text

make(1): error out if an '.endif' contains extraneous text

make(1): make structure of the code in Cond_EvalLine clearer

make(1): don't pretend to expand CondParser_Eval

At that point, the variable expression has already been expanded. To
avoid the impression that the token might be relevant, pass FALSE
instead of TRUE. No change of behavior.

make(1): remove const from function parameters

These have been left-overs from refactoring, when these pieces were
extracted to separate functions.

make(1): remove redundant assignments

make(1): reduce memory allocation for dirSearchPath

make(1): reduce memory allocation for CmdOpts.create

make(1): remove pointer indirection from GNode.commands

Just to save a few memory allocations. No noticeable effect on the
performance though.

make(1): use properly typed comparisons in boolean contexts

make(1): align end-of-line comments with tabs

make(1): indent cond.c with tabs instead of spaces

make(1): in lint mode, check for ".else <cond>"

make(1): use bitset for IfState

Previously, the individual IfStates contained redundant information,
which was apparent in the documentation. This led to expressions like
(state > ELSE_ACTIVE) that are hard to read since the reader has to look
up the order of the enum.

To avoid this, the state of an '.if' block is now encoded using a bitset,
encoding the properties of each state directly. This replaces the
previous (state > ELSE_ACTIVE) with !(state & IFS_ACTIVE), which is
easier to understand.

No change in behavior.

make(1): replace large switch with if-else in Cond_EvalLine

make(1): move the comment about ".else <cond>" to the correct place

make(1): improve readability of IfState documentation

The words "None" and "One" looked and sounded too similar.

make(1): rename local variable in Cond_EvalLine

A variable called line is not supposed to be a moving target, that's what
the name p is reserved for.

make(1): remove complicated code to find the definition for plain .if

make(1): document IfState in more detail

make(1): clean up Cond_EvalLine

The constant MAXIF was not a maximum but an initial capacity. Inline it
to remove the misleading name. Likewise, MAXIF_BUMP was an unnecessary
and wrong name.

Rename the enum since it only describes a single state, not multiple.

Rename the stack of states since it describes multiple states, not one.

Add markers where to add the missing error messages for unknown
directives or extraneous cond, such as in ".else cond" or ".endif cond".

make(1): document non-obvious special case in EvalNotEmpty

make(1): add test to cover the If_Eval call in EvalNotEmpty

make(1): remove unreachable code from CondParser_String

The code that has been removed was not reachable from after the Var_Parse
call. All branches after the Var_Parse are either freed properly or
return the expanded value.

make(1): fix error message when parsing unknown .if (since 2005-05-01)

make(1): add tests for parsing conditionals, document a parsing bug

make(1): document ParseEmptyArg

make(1): remove unnecessary NULL marker from functions list

Comparing the pointer works equally well and uses less memory.

make(1): document handling of '#' and '\n' in conditionals

make(1): in lint mode, only allow '&&' and '||', not '&' and '|'

These variants of the condition operators are neither documented in the
manual page nor are they used in practice.

make(1): extract CondParser_Func from CondParser_LeafToken

make(1): inline strchr call in CondParser_LeafToken

make(1): rename CondParser_Func to CondParser_LeafToken

The previous name was misleading since the function did not only parse
function call expressions.

make(1): fix type mismatch between int and Token

make(1): improve local variable name in EvalNotEmpty

make(1): improve variable names in EvalNotEmpty

In a unary expression, there is no left-hand side.

make(1): rename local variable in CondParser_String

make(1): clean up TryParseNumber in conditions

More descriptive variable names, more appropriate literals for
comparisons, one task per paragraph of code.

make(1): clean up FuncExists

make(1): clean up code related to VarEvalFlags

Mention VARE_WANTRES before VARE_UNDEFERR since the latter depends on
the former.

In ApplyModifier_Assign, VARE_KEEP_DOLLAR doesn't have to be removed
from eflags since ParseModifierPart does this already.

In EvalUndefined, testing for VARE_WANTRES is redundant if VARE_UNDEFERR
is already set.

make(1): clean up VarEvalFlags in the calls to Var_Parse and Var_Subst

There are only 3 flags, and some combinations don't even make sense.

VARE_UNDEFERR only makes sense if combined with VARE_WANTRES. If the
latter is not set, the variable expressions are only parsed, without
asking whether they are defined or not. Therefore, VARE_UNDEFERR cannot
have any effect in that case.

VARE_KEEP_DOLLAR is actively ignored by ParseModifierPart. In cases
where VARE_WANTRES is not set, this means that VARE_NONE can be passed,
which is easier to grasp than any bitset operations. This also gets rid
of a few type casts from enum to unsigned int that were necessary to
pass WARNS=6.

make(1): clean up CondParser_Comparison

make(1): reword condition in CondParser_String

The parentheses were confusing for human readers.

The compiler doesn't really care about the wording of the condition, GCC
5 on amd64 generates non-obvious but nice code anyway, replacing the
logical or with a logical and.

make(1): fix type mismatch between Boolean and Token

Even though the C boolean and Token are encoded the same, the code should
still distinguish them, for the benefit of human readers.

make(1): fix type mismatch in If_Eval

No practical consequences since TOK_FALSE == 0 and TOK_TRUE == 1.

make(1): add test for parse error in function defined()

make(1): add test for parse error in condition using empty()

make(1): make API of Buf_Init simpler

In most cases, the caller doesn't want to specify the exact number of
preallocated bytes.

make(1): clean up code stylistically

* Replace character literal 0 with '\0'.
* Replace pointer literal 0 with NULL.
* Remove redundant parentheses.
* Parentheses in multi-line conditions are not redundant at the
beginning of a line.
* Replace a few !ptr with ptr == NULL.
* Replace a few ptr with ptr != NULL.
* Replace (expr & mask) == 0 with !(expr & mask).
* Remove redundant braces for blocks in cases where the generated code
stays the same. (Assertions further down in the code would get
different line numbers.)
* Rename parameters in CondParser_String to reflect the data flow.
* Replace #ifdef notdef with #if 0.

The generated code stays exactly the same, at least with GCC 5.5.0 on
NetBSD 8.0 amd64 using the default configuration.

make(1): merge duplicate code for skipping horizontal whitespace

make(1): fix tests directive-ifmake and varcmd with custom boolean

When make is compiled with -DUSE_UCHAR_BOOLEAN, these tests failed.
Merge duplicate code and don't depend on the actual value of TRUE when
evaluating conditions.

make(1): remove redundant parentheses from sizeof operator

The parentheses are only needed if the argument is a type, not an
expression.

make(1): remove word "Ptr" from variable names

Whether or not a variable is a pointer is obvious from the context.
Since the introduction of function prototypes in C90, this information
is checked by the compiler and no longer needs to be encoded in the
variable names.

make(1): change char * to void * in Var_Value

The only purpose of the parameter freeIt is to free the memory
associated with the return value. To do this, no pointer arithmetic is
needed. Therefore, change to a void pointer, to catch accidental use of
that pointer.

make(1): clean up is_separator

make(1): fix parse error in string literal in conditional

The string literal "${VAR} " had produced a "Malformed conditional", at
least since 2003. (That's the oldest make I have available for testing.)
Strange that nobody else noticed that in the last 17 years.

make(1): rename VAR_CMD to VAR_CMDLINE

Since make has to do with both the command line and child commands, the
former name was confusing.

make(1): group the command line options and arguments

By having a single struct that holds all command line options and
arguments, it is easy to see in the code when such a command line
argument is modified. It also cleans up the namespace since the command
line options don't follow a common naming style. Having them in a
struct also means that there is a single place for putting the
documentation, not two as before.

The struct also suggests to extract the initialization code out of main,
which is still too large, having more than 400 lines of code and
covering far too many topics.

make(1): clean up comments in cond.c

make(1): negate OP_NOP and rename it to GNode_IsTarget

make(1): inline CondFindStrMatch into FuncMake

make(1): remove redundant type casts

This mainly affects the void pointers in callback functions for lists.
These had been necessary once when the parameter type was still
ClientData instead of void pointer.

make(1): add tags to enum types

This allows IDEs to offer better type information than "anonymous enum".

make(1): fix indentation

make(1): make cond.c ready for WARNS=6, part 2

make(1): make cond.c ready for WARNS=6

make(1): revert previous commit

It had accidentally reverted all the work from the past few days.

make(1): fix double-free bug in -DCLEANUP mode (since 2020-10-02)

The bug had been introduced with dir.c 1.155 on 2020-10-02 22:20:25. In
that commit, openDirectories was replaced with a combination of a list
with a hash table, for more efficient lookup by name.

Upon cleanup, OpenDirs_Done is called, which in turn called
Dir_ClearPath. Dir_ClearPath takes full ownership of the given list and
empties it. This was no problem before since afterwards the list was
empty and calling Lst_Free just frees the remaining list pointer.

With OpenDirs, this list was combined with a hash table, and the hash
table contains the list nodes, assuming that the OpenDirs functions have
full ownership of both the list and the hash table. This assumption was
generally correct, except for the one moment during cleanup where full
ownership of the list was passed to Dir_ClearPath, while the hash table
still contained pointers to the (now freed) list nodes. This by itself
was not a problem since the hash table would be freed afterwards. But
as part of Dir_ClearPath, OpenDirs_Remove was called, which looked up
the freed directory by name and now found the freed list node, trying to
free it again. Boom.

Fixed by replacing the call to Dir_ClearPath with code that only frees
the directories, without giving up control over the list.

make(1): fix test for .ifndef when compiled with -DUSE_UCHAR_BOOLEAN

In that compilation variant, TRUE is defined to 255, to see whether all
boolean expressions evaluate to either 1 or 0. The field If.doNot in
cond.c doesn't do this since it uses the actual value of TRUE.
Therefore, change the evaluation slightly to also handle this case.

make(1): use consistent pattern for parsing whitespace

The pp and cpp in the function names stand for "parsing position" and
"const parsing position".

make(1): remove redundant function prototypes

make(1): replace += 1 with ++ and -= 1 with --

Just for visual consistency. The generated code stays exactly the same.

make(1): make debugging code shorter

make(1): normalize whitespace in source code

There is no more space tab. Either only tabs or only spaces or tabs
followed by spaces, but not spaces followed by tabs.

make(1): clean up API for finding and creating GNodes

The previous API had complicated rules for the cases in which the single
function returned NULL or what it did. The flags for that function were
confusing since passing TARG_NOHASH would create a new node even though
TARG_CREATE was not included in that bit mask.

Splitting the function into 3 separate functions avoids this confusion.
It also reveals several places where the complicated API led to
unreachable code. Such code has been removed.

make(1): rename variables cp2 to be more expressive

make(1): add tags to some of the unnamed structs

The tags prevent the structs from accidentally becoming compatible
types.

While here, remove a few typedefs for structs that are single-purpose,
since there is no point in abstracting from the actual representation of
these types.

make(1): rename local variables in ParseFuncArg

make(1): rename VarParseErrors to VarParseResult

The name VPE_OK was confusing since it was not an error at all.

make(1): fix coverity annotation for CondParser_String

It had been broken since 2020-09-08, when CondLexer (now CondParser) was
added.

make(1): suppress wrong "Malformed conditional" for undefined variables

This only has an effect in lint mode right now.

make(1): prepare Var_Parse for proper error handling and reporting

Right now, Var_Parse swallows many errors during parsing and evaluation.
Ideally, these errors should propagate from the deeply nested
expressions where they occur up to the top-level expressions. When such
an error occurs, the depending expressions should not be evaluated any
further. They may still be parsed, but side effects should be
minimized.

The goal is to prevent incomplete expressions like the "xy}" in
moderrs.exp:106 from being evaluated and eventually passed to the shell
for execution. This expression is a left-over from a parse error in the
mod-t-parse target in moderrs.mk:154.

This commit is a first step in analyzing and verifying the current state
of affairs. The modelling in VarParseErrors already looks complicated
but is expected to closely match reality.

make(1): clean up RCSID blocks

These blocks mostly consisted of redundant structure, following the same
#ifndef pattern over and over, with only minimal variation.

It's easier to maintain if the common structure is only written once and
encapsulated in a macro.

To avoid "defined but unused" warnings from GCC in the case where
MAKE_NATIVE is not defined, I had to add volatile. Adding
MAKE_ATTR_UNUSED alone would not preserve the rcsid variable in the
resulting binary.

make(1): fix parameter type of CondEvalExpression

make(1): clean up API for evaluating conditions

There was no need to make struct If publicly visible.

There was no need to have parameters in the public API that were passed
the same constants all the time.

The former function names had not been distinctive.

make(1): rename Var_ParsePP back to Var_Parse

The migration to the "parsing position" pointer has been done.

make(1): migrate CondParser_String to Var_ParsePP

make(1): use correct character literals in TryParseNumber

make(1): clean up code in CondParser_String

The two parts of the for loop were not related in any way, which made
the code more confusing than necessary.

make(1): fix inconsistent code indentation

make(1): inline local variable in Cond_Eval

make(1): fix wording of parse error in conditionals

The word "should" is not appropriate for situations that eventually lead
to a parse error.

make(1): add wrappers around ctype.h functions

This avoids casting the argument to unsigned char, and to cast the
result of toupper/tolower back to char.

make(1): split EvalComparison into smaller functions

make(1): clean up code for evaluating conditions

make(1): extract EvalNotEmpty from CondParser_Comparison

make(1): split CondParser_Comparison into 2 parts

make(1): document CondParser_Comparison

make(1): add tests for really strange edge cases in conditions

make(1): reorder parameters of condition parsing functions

First the subject, then the options, then the output parameters.

make(1): reduce code size in CondParser_Eval

make(1): use consistent naming scheme for condition parsing functions

make(1): rename CondGetString to CondParser_String

This describes the function's effect more accurately. The verb "get" is
not commonly associated to having side effects.

make(1): replace *par->p with par->p[0]

It's a few characters more code than before but can be read strictly
from left to right, which was not possible before.

make(1): rename CondParser.condExpr to p

The prefix "cond" was needed when this struct field was a global
variable. The main name "expr" was not precise enough since this code is
about parsing a condition, not an expression.

During parsing, this variable does not contain the whole expression but
a pointer to the remaining part of the condition, therefore the name
"expr" had been confusing.

make(1): rename CondLexer to CondParser

The name CondLexer was wrong since this type is about parsing conditions
that can be arbitrarily nested.

make(1): reduce code size in CondGetString

The pattern is now the usual "test the character, then increment the
pointer", throughout the whole function.

make(1): skip redundant condExpr-- in CondGetString

make(1): reduce code size for parsing the || or && operators

On x86_64, accessing [0] generates less code than [1].

make(1): clean up comments in cond.c

make(1): add CondLexer_SkipWhitespace

make(1): rename CondLexer.condPushBack to curr

This variable has some context information from the struct now,
therefore its name can be shorter.

make(1): group the condition parsing state into a struct

Instead of having 3 global variables, the struct clearly communicates
that the 3 variables belong together. During debugging, it's easy to
just "p *lex" instead of remembering the names of the 3 former global
variables.

Converting the global variables into a local variable makes it
immediately clear that the functions in this file operate on this
struct. Keeping the global variables in mind is more difficult. Having
a local variable also gets rid of the 3 sv_* variables in
Cond_EvalExpression, which were also a sign that these "global
variables" were not that global at all.

This commit only contains the minimal code changes for converting the
variables into a local struct. It was tempting to add functions like
CondLexer_SkipWhitespace, but this is better left for a follow-up
commit.

make(1): in CondGetString, replace repeated Buf_Add with Buf_AddStr

make(1): rename local functions for parsing conditions

The word "get" implies a cheap operation without side effects. Parsing
instead has lots of side effects, even if it's only that the parsing
position is updated.

make(1): migrate get_mpt_arg to Var_ParsePP

This part is covered well by the unit tests. When I forgot to decrement
the linePtr, several of them failed reliably.

make(1): migrate Var_Parse in CondGetArg to Var_ParsePP

make(1): document use of magic values in CondDoEmpty

make(1): update documentation for Cond_EvalExpression and Cond_Eval

make(1): make parameter of Cond_Eval and Cond_EvalExpression const

make(1): trust that Var_Parse never returns NULL

That function is quite long, but all its return paths lead either to the
expanded variable expression, or to var_Error or varNoError.

make(1): rename Lst_FindB back to Lst_Find

The migration from "comparison function" to "match function" is done,
the "B" in the names is no longer needed.

make(1): start replacing Lst_Find with Lst_FindB

Lst_Find is called with a "comparison" function that returns the integer
0 if the desired node is found. This leads to confusion since there are
so many different return value conventions for int, such as 0/1 for
mimicking false/true, -1/0 as in close(2), and the sign as in strcmp(3).
This API is much easier to understand if the "comparison" function is
not called a comparison function (since that is too close to strcmp),
but a "match" function that just returns a boolean.

In Lst_FindFromB, the node argument may be null. This deviates from the
other Lst functions, which require Lst and LstNode to generally be
non-null. In this case it is useful though to make the calling code
simpler.

In arch.c, this makes a lot of the previous documentation redundant.

In cond.c, the documentation is reduced a little bit since it had
already been cleaned up before. It also removes the strange negation
from CondFindStrMatch.

In dir.c, the documentation collapses as well.

In main.c, separating the ReadMakefile function from the callbacks for
Lst_FindB allows the former to get back its natural function signature,
with proper types and no unused parameters.

To catch any accidental mistakes during the migration from Lst_Find to
Lst_FindB, the code can be compiled with -DUSE_DOUBLE_BOOLEAN, which
will complain about incompatible function pointer types.

make(1): remove trailing 'S' from names of Lst functions

The migration from null-passing Lst functions to argument-checking Lst
functions is completed.

There were 2 surprises: The targets list may be NULL, and in Dir_AddDir,
the path may be NULL. The latter case is especially surprising since
that function turns into an almost-nop in that case. This is another
case where probably 2 independent functions have been squeezed into a
single function. This may be improved in a follow-up commit.

All other lists were fine. They were always defined and thus didn't
need much work.

make(1): migrate Lst_Find to Lst_FindS

make(1): migrate Lst_IsEmpty to Lst_IsEmptyS

make(1): reverse order of the Lst_Find parameters

The other callbacks all have (function, param), only the Lst_Find had
(param, function), which was inconsistent.

make(1): clean up headers

Remove redundant headers that are already included by "make.h".
Make <assert.h> available to all compilation units that use "make.h".

make(1): move complicated boolean expression out of the function call

It's easier to inspect in a debugger this way.

make(1): fix type of string length variables

make(1): fix wrong or outdated comments

make(1): fix type mismatch in compare_expression

This was caught by a simple "typedef double Boolean" in sprite.h.

If only there were a C compiler with "strict" mode, where pointers,
numbers and booleans were separate and unassignable data types, this
copy-and-paste typo would have been caught much earlier.

make(1): fix type of local variable in CondGetString

When the Boolean type from sprite.h is replaced with C99's standard bool
type, the chained assignment leads to the following compile error with
GCC 5:

cond.c:404:5: error: suggest parentheses around assignment used as truth
value [-Werror=parentheses]
*quoted = qt = *condExpr == '"' ? 1 : 0;

Changing the type of qt from int to bool fixes this.

make(1): format the source code consistently, at least per file

Some files use 4 spaces per indentation level, others use 8. At least
for the few files from this commit, they use a consistent style
throughout each file now.

In Cond_Eval, the #define has changed into an enum since the identifiers
need not be visible to the C preprocessor.

make(1): remove trailing Z from buffer functions

This Z had been useful during the migration from int to size_t. This
migration is finished, at least for the Buffer type, so the Z is no
longer necessary.

make(1): mark condition strings as constant

make(1): remove redundant documentation from cond.c

Many of the functions in cond.c are so small that it's faster to read
their code instead of a large block of documentation.

There are other functions that actually need a detailed description.
These have been left as-is.

make(1): remove redundant function prototypes from cond.c

make(1): no declaration-after-statement anymore

NetBSD make is intended to be maximally portable, therefore it uses only
C89. This was not declared in the Makefile before.

There are still a few places in parse.c and metachar.c that use
end-of-line comments. These will be fixed in a follow-up commit.

make(1): switch Buffer size from int to size_t

This change helps to make the various integer types compatible and is a
preparational step for setting WARNS=6 in the Makefile.

The documentation of buf.c has been cleaned up and condensed since it
was mostly redundant, and some statements were even slightly wrong.

All code changes are covered by the existing unit tests, except for the
few lines in for.c around for_var_len. These changes have been reviewed
thoroughly and manually, like all the others in this commit.

Those buffer functions that deal with sizes have been renamed by
appending a Z, to make sure that no function call was accidentally
forgotten. They will be renamed back in a follow-up commit.

As usual, the scope of a few affected variables has been reduced, and
some variables had to be split since they had been incorrectly merged
before.

The order of the arguments to Buf_AddBytes has changed from (mem_len,
mem) to (mem, mem_len), in order to make it consistent with the
functions from the C standard library, such as snprintf.

make(1): use enum for return values of Cond_Eval and friends

make(1): use consistent indentation in source code

Tabs for multiples of 8, then spaces.

The usage string has been kept as-is since the spaces there are
indentional and do influence the output.

make(1): avoid calls to free(3) in the common case of a NULL pointer

make(1): make CondDoDefined simpler

make(1): make return value of Var_Parse constant

This return value is not supposed to be modified since it can be a string
literal. The modifiable part is returned via freePtr, but only for
freeing, not for actually modifying anything.

make(1): clean up unnecessary snprintf and multi-line function calls

make(1): rename Varf_Flags to VarEvalFlags

In var.c there are lots of different flag types. To make any accidental
mixture obvious, each flag group gets its own prefix.

The only flag group that is visible outside of var.c is concerned with
evaluating variables, therefore the "e", which replaces the former "f"
that probably just meant "flag".

compare_expression: return after fetch lhs and rhs if !doEval

Otherwise we end up throwing warings/errors for valid
conditionals due to not expanding variables fully.

Add tests to catch this.

Reviewed by: rillig

make(1): remove trailing whitespace

make(1): remove redundant parentheses around return values

make(1): fix evaluation of unreachable conditions

Since 2015-10-11, make had evaluated unreachable conditions even though
the manual page said it didn't.

Replace dead conditional by assert.

CID 975995

branches: 1.74.2; 1.74.4;
Collapse the 3 boolean parameter to 1 flags parameter. No functional change.

remove free NULL checks (Tilman Sauerbeck)

Preserve $$ in := assignments..

FOO=\$$CRAP
BAR:=${FOO}

all:
echo ${FOO}
echo ${BAR}

Fix evaluation of defined(FOO) and ${FOO} > 0
add a unit-test to catch it.

Avoid accessing beyond end of empty string.

Detected by Address Sanitizer - dim at FreeBSD.org

Add Boolean wantit to Var_Parse and Var_Subst

wantit will be FALSE when we are just consuming to discard
in which case we skip "expensive" things like Cmd_Exec.

Reviewed by: christos

When evaluating condtionals from .if we want to require
that the lhs is a variable reference, a number or a quoted string.
This helps avoid subtle bugs caused by typos.

When conditionals are being evaluated during variable expansion
we cannot be as strict becuase lhs will already have been expanded.

We therefor pass a boolean to Cond_EvalExpression to tell it how
lhs should be treated.

Add unit-tests/cond2.mk to test the above

Reviewed by: christos, joerg

off-by-one

Fix build break due to signed/unsigned comparison

Allow cond_state[] to grow.
The need is rare, but real.

Reviewed by: christos

branches: 1.64.2;
Replace __dead, __unused and the various printf format attributes
with versions prefixed by MAKE_ATTR_* to avoid modifying the
implementation namespace. Make sure they are available in all places
using nonints.h to fix bootstrap on Linux.

In meta mode, level 0 can legitimately see > 64 nested .if's

branches: 1.62.4;
CondDoExists: Avoid use after free()

Mark unused args.

branches: 1.60.2;
Change enum values so that TOK_FALSE is 0 and TOK_TRUE is 1.
Use this fact to remove loads of ? : clauses.

Rename all the members of 'enum Token' to TOK_FOO.
Makes it rather more obvious wherethey belong - especially since
two of them were 'True' and 'False' (and not 1 and 0 either).

Treat .ifdef ${foo} as .if defined(${foo}) not .if "${foo}" != "".
(and similarly for the other .ifxxx).
Do comparison against "" or 0 for .if "foo" and .if <numeric> directly
instead of faking up the operator string.
Rename error: to done: and use it for more exit paths.
Elucidate and correct some comments.

Fixes problems with makefiles that do:
.for var in var_1 var_2
.ifdef ${var}
...
which used to check whether var_1 was defined - because the .ifdef saw the
literal var_1, not a variable expansion due to the way .for loop variables
were substituted.

Avoid infinite loop.

Allow for () in the argument to .ifdef et al.
Save/restore current values of global data across expression evaluation
to give ${foo:? ...} a change of working inside another .if.

Sprinkle some const.
In particular for Lst_Find() and Lst_FindFrom().
Remove some unneeded casts and some now-undeeded UNCONST().

Change 'ClientData' to 'void *' so that relevant parameters can
be made 'const void *'.

There is no point in saving the last character of a NUL-terminated
string in order to overwrite with a NUL, then restore the original value!
Remove and simplify the CondDoFoo() functions.

Revert previous - cause grief with .ifdef (foo)
Instead return False for exists(), defined(), make() etc when the
argument has length zero.

Only terminate a conditional function argument on ')' if we expect the
argument to be exclosed in (...).
'.if exists()' is parsed as '.ifdef exists()' and the ')' must not be
left over at the end.
I have no idea why any of my recent changes have affected this.
But pkgsrc/mk/compiler/gcc.mk (line 488) does .if exists(${FCPATH})
even though FCPATH is only set when the file exists.

Change 'Buffer' so that it is the actual struct, not a pointer to it.
Saves having to malloc/free a fixed size structure.
Buf_Init() now takes ptr to Buffer to initialiase.
Change Buf_Destroy() to return ptr to string when not freed.
Remove large number of casts to (Byte) and (Byte *) - 'Byte' is 'char' here.
Buf_AddByte[s] guarantees that the data is 0 termininated, so never add '\0'.
Keep 'count' not 'left' and 'inPtr', code is simplier with only one update.
Fix fallou, no functional change.

Use NULL instead of -1 cast to the relavant type (usually via NIL).
This was a suggestion from christos - so blame him if there is a deep
reason for using -1 :-)

Before deciding to use the default function in a .if, check whether the
token that follows the argument might be '==' or '!='.
If so then treat as a string comparison instead.
Fixes bin/15233 and bin/30967 provided some whitespace is present.
".if A==A" remains a check for defined(A==A) since make places no
restrictions on the names of variables!

Common up parsing .if defined() make() exists() target() commands() and empty()
Add a check for the '(' following the function name, if absent then treat
as if the function name is unknown - usually leading to a syntax error.
No other functional changes intended.

Do a string compare if the rh argument doesn't completely convert to a number
(instead of silently ignoring the chars that failed to convert).
Use strtoul() instead of homebrew copy.
Only use strtod() if strtoul() fails because the value is too large or
contains '.', 'e' or 'E'.
Do a compare for strings that start '-' or '+' as well as digits.

Fix conversion of hex numerics in comparisons.
Broken by a fix from christos 14 years ago.

Move two great chunks of code out of a switch statement and into
separate functions.
No functional change.

If there is a syntax error in a .if expression then ignore the entire
conditional block (ie down to the matching .endif) instead of passing
the input line back to the normal parsing code.
Do the same if the maximum .if nesting (64 levels) is breached.

If Parse_Error is called after curFile is invalid, set it to a dummy
to avoid seg-fault. Also the NULL filename will cause ParseVErrorInternal
to skip trying to report file and line number info.

If CondToken is called outside the context of a .if etc, variables in
the expression being parsed will already be expanded, so allow for
an unqouted number to jump us back into the compare logic.
This allows ${${SomeNumber:U42} > 0:?True:False} to work anywhere.

Job_Finish should process postCommands if it has commands or children.

back all changes out until I fix it properly.

- use pid_t/size_t as appropriate instead of int.
- use %ld to print pids.
- fix a bit of lint.
- WARNS=4

Print the complain about random stuff after empty only once.

Warn for constructs like
.if empty xxx(foo)
which are interpreted as
.if empty(foo)
or
.if empty (foo)

This is based on rillig's commit to pkgsrc/devel/bmake, but doesn't warn
for the third case. Discussed with sjg.

branches: 1.37.4;
Add code to ensure that .if/.endif are correctly nested with .for/.endfor
and also include files don't have mismatched .if/.endif
It has been suggested that make used to have this test, but I can't
remember removing any code that might have performed it.

Simplify the code that processes .if lines.
It doesn't need a two-dimensional array to remember the states of .if lines.
It would be even simpler if we didn't try to detect .else and .elif lines
that follow .else lines.
Unfortunately this isn't the code that is stupendously slow...

branches: 1.35.2;
Since 'ClientData' is 'void *', nuke almost all the (ClientData) casts.

Output all debug trace output through 'debug_file' defaulting to 'stdout'.
(Almost all the debug output went there, but some went to stderr.)
Split the parsing of -d (debug flags) out into its own routine.
Allow the output filename to be changed by specifying -dF<file> to create
a log file, or -dF+<file> to append to it. <file> may be stdout or stderr.
Also change so that -d-<flags> acts on <flags> locally but doesn't copy
them to MAKEFLAGS so they aren't inherited by child makes.
I'm not 100% happy with the command line syntax for the above, so they are
currently undocumented.

Add debug output for CondDoExists() - the results can be surprising.

Coverity CID 526: Check for condTop < 0 where we decrement it, not later
when the damage is already done.

Remove bogus debugging code I accidentally committed.

Add some coverity allocation comments, and change the way the allocator
functions work. When they allocate storage that needs to be freed, instead
of setting a boolean, set the pointer to be freed. Plug some more memory
leaks found by inspection.

From Max Okumoto:
- Remove casts to NULL.
- Remove space between cast and object.

Whitespace KNF cleanup from Max Okumoto

appease gcc -Wuninitialized

When parsing conditional tokens, make sure that the token is followed by
a non-alpha character, so that .elsefoo is not parsed as .else leading
to confusion later.

PR/29203, PR/29204: Max Okumoto: KNF changes to make [no functional changes]

Simplify build, no functional changes.

Instead of adding MAKE_BOOTSTRAP for hosted environments, i.e., when
you want things simple, instead add MAKE_NATIVE to get those hugely
important features like __RCSID().

It's now possible to build make on some hosts with: cc *.c */*.c

Allow "string" and "${varspec}" on lhs of conditionals.
This makes it possible to use expressions like
${("${LIST:Msomething"}):?present:absent}
it also makes treatment of lhs and rhs more consistent, and
makes CondToken easier to read.

Update unit-test/cond1 to test new features.

Revised patch for catching extraneous .else's
We actually need to tack condTop and skipIfLevel.
Bump MAXIF while here.
Enhance the unit-test for conditionals to exercise more code.

PR: 24420

Back out previous patch for now.
Breaks bsd.obj.mk

Add check for extraneous .else's - based on patch from Arne H. Juul
in PR/24420.
Add a unit-test for conditionals.
Also in the unit-test makefile strip any .CURDIR in output.

PR: 24420

branches: 1.19.2;
Handle more complex conditional expressions.
CondCvtArg now returns NULL if it consumed all input, or a pointer
to any left overs.

Make empty() consider an undefined variable as empty,
rather than throw a syntax error.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22365, verified by myself.

Pass WARNS=3

math.h is not needed and including it introduces uncessary dependencies
on some systems.

PR: 21204

Treat an unquoted '#' as end of line when parsing conditionals.
This ensures that comments are ok at the end of line.

PR: 19596
Reviewed by: christos

Remove !__STDC__ stuff, de-__P(), ANSIfy, and de-register.

Provide a new .if commands(<target>) primitive that returns true if the
target has commands associated with it.

Fix conditional variable expression parsing (reported by cgd)

Add conditional variable evaluation from Alistair Crooks.

branches: 1.9.2;
wrap #include <sys/cdefs.h>, __RCSID(...) stuff in #ifndef MAKE_BOOTSTRAP

Add WARNS=1
RCSID police

- Merge in FreeBSD and Lite2 changes.
- Fix bug where a non-archive target with a .a suffix would always
be considered to be out of date, since it does not have a TOC.

branches: 1.6.6;
- $NetBSD$ rcsids
- Fixed so that .[A-Z]* targets that do not match keywords are ignored as
Posix mandates
- Added .PHONY target keyword

Fixes from Christos Zoulas, who used purify, objectcenter and testcenter
to find memory leaks and illegal memory accesses.

fixes/improvements from Christos Zoulas <christos@deshaw.com>.

Include appropriate header files to bring prototypes into scope.

Add RCS identifiers.

branches: 1.1.1;
Initial revision

make: fix a few more memory leaks

make: clean up comments, code and tests

make: remove redundant comments

No functional change.

make: refactor CondParser_Term to be inlinable

No functional change.

make: clean up parsing of conditions

No functional change.

make: clean up comments

No binary change, except for line numbers in assertions.

make: simplify memory allocation for string buffers

In edge cases and short-lived buffers, the initial buffer size is
irrelevant, so use the default.

No functional change.

make: clean up comments

No binary change, except for line numbers in assertions.

make: clean up names of local variables

No binary change.

make: replace 'variable expression' with 'expression' in comments

No binary change.

make: clean up multiple-inclusion guards

No functional change.

make: clean up variable and function names

No functional change.

make: warn about malformed patterns in ':M', ':N' and '.if make(...)'

These patterns shouldn't occur in practice, as their results are tricky
to predict. Generate a warning for now, and maybe an error later.

Reviewed by sjg@.

Allow guard targets to use variables.

I commonly use __${.PARSEDIR:tA}__ where a unique guard
is needed, __${.PARSEDIR}__ is also useful in many cases.

Combination of patch from rillig and mine

make: allow targets to be used as multiple-inclusion guards

This style is used by FreeBSD, among others.

make: add tests for full code coverage of multiple-inclusion guards

make: clean up code for skipping files with multiple-inclusion guard

No functional change.

make: if a makefile is protected by a guard, only include it once

"looks reasonable" sjg@

make: remove parameter names from function declarations

No binary change.

make: shorten function names, clean up comments

No functional change.

make: reduce complexity of evaluating expressions

No functional change.

make: don't interpret the return value of Var_Parse

The return value of Var_Parse is largely redundant to the returned
string. The idea behind the type VarParseResult was to migrate all call
sites to checking this return value instead of the returned string, but
that hasn't happened. Instead, the additional type only added more
complexity.

There was a single place where that return value was actually used, when
parsing conditions. And even in that case, ignoring the VarParseResult
added back an error message that previously hid bugs, in the test
cond-token-plain.mk.

Even though these error messages are redundant in the other tests, they
don't hurt as they don't happen often.

make: clean up tracking of depth of nested .if directives

The variable cond_min_depth was redundant. It was only accessed while
parsing the makefiles. Merging it into struct IncludedFile removes the
possible confusion between cond_min_depth and including_cond_min_depth.

No functional change.

make: fix variable and function names relating to .if nesting

The previous names were confusing since they suggested that cond_depth
instead of cond_min_depth would be saved and restored.

No functional change.

make: move Cond_save_depth above Cond_restore_depth

This puts the functions into chronological order, as saving happens
before restoring.

No functional change.

make: use assertion for internal error condition

When the nesting level of conditionals is restored to an unreasonably
high value, the error message "0 open conditionals" doesn't make sense.

Cond_reset_depth just use cond_min_depth

To avoid errors from unclosed conditionals on .break
it is sufficient to just set cond_depth = cond_min_depth.

Patch from rillig

make: list comparison operators in declaration order

This allows a tiny optimization in the switch statement.

No functional change.

make: add more details to error message for numeric comparison

Before:
String comparison operator must be either == or !=

After:
Comparison with '>=' requires both operands 'no' and '10' to be numeric

Noticed by martin@ in pkgsrc/textproc/py-pygments.

make: add .break to terminate .for loop early

When .break is encountered within a .for loop
it causes immediate termination.

Outside of a .for loop .break causes a parse error.

Reviewed by: christos

tests/make: adjust expectations to actual behavior

The bug in deptgt-silent-jobs.mk has been fixed, the debug logging for
comparing conditions and for deleting global variables has changed
intentionally.

make: improve comments about parsing and evaluating conditions

No binary change.

make: improve local variable name in ParseWord in conditions

That function not only parses function arguments but also bare words, so
the name argBuf didn't match anymore.

No binary change.

make: make debug logging for comparisons less technical

make: simplify control flow in CondParser_Comparison

No functional change.

make: fix mistakes, spelling and typos in comments and manual page

No binary change for -DNDEBUG.

make: clean up variable names

No binary change.

make: rename labels in CondParser_Leaf

No binary change.

make: replace Var_Value with Var_Exists where applicable

The latter function already existed in 1993, no idea why it was not
used.

No functional change.

make: inline EvalBare

No functional change.

make: clean up function argument names and comments

No binary change except for assertion line numbers.

make: remove redundant initializer in CondParser_ComparisonOrLeaf

No binary change.

make: clean up nitpicks

In ParseWord, the expressions '*p' and 'ch' are the same.

In ParseDependencyTargetWord, clean up a wordy comment and join two
conditions.

In the test cond-token-number, clarify that make doesn't convert from
hex to decimal but only from hex to internal representation.

No functional change.

tests/make: demonstrate edge case that evaluates an expression twice

make: remove redundant code from CondParser_ComparisonOrLeaf

No functional change.

make: split ParseWord into the actual ParseWord and ParseFuncArg

Combining two similar but fundamentally different parsing tasks in a
single function only increased the complexity, of the implementation as
well as the call sites.

The code makes it obvious now that a function argument is a bare word
surrounded by parentheses.

The special case of an empty word is only needed for the function
argument, it cannot occur in a bare word. The code for that has been
moved to the caller. Such an empty word not only occurs for 'defined()'
but also for 'defined(${:U})'.

No functional change.

make: make ParseWord in condition parser simpler

Merge the two return values (bool, string) into a single return value.
As before, the caller cannot observe the difference between a parse
error and an empty word, both are handled in the same way.

In CondParser_ComparisonOrLeaf, the word cannot be empty since the
calling function CondParser_Token already handles all cases that could
lead to an empty word.

No functional change.

make: internally return false for irrelevant leaves in conditions

The result of irrelevant leaves is effectively ignored by CondParser_And
and CondParser_Or. Use the 'doEval &&' pattern to make the code
consistent with CondParser_Comparison and CondParser_FuncCall.

No functional change.

make: in irrelevant function calls in conditions, return false

When a condition contains an irrelevant function call, it doesn't matter
whether the function call evaluates to true or to false, it will be
discarded anyway by either CondParser_And or CondParser_Or.

Returning false instead of true makes the code simpler, plus it is more
common to return false for irrelevant results.

No functional change.

make: use simpler return type for ParseWord in conditions

No functional change.

make: replace table for function lookup in conditions with simple code

The code for looking up the function from the table forced the compiler
to use a specific memory layout. Replacing the table with explicit code
provides the compiler more opportunities to optimize the code. Another
side effect is that there are fewer pointer operations.

Previously, is_token checked that the character after the word does not
continue the word, this is now done separately since for the function
lookup, this check was unnecessary. The newly added skip_string
provides a higher abstraction level, it is no longer necessary to pass
the string length as a separate, redundant parameter.

No functional change.

make: clean up condition parser

No functional change.

make: merge duplicate types CondEvalResult and CondResult

No binary change.

make: merge duplicate enum constants for CondEvalResult and CondResult

No binary change.

make: remove redundant parameter for evaluating conditions

No functional change.

make: merge types CondResult and CondEvalResult

No functional change.

make: rename local variables to be simpler

No binary change.

make: clean up comments

make: format comments according to /usr/share/misc/style

Assisted by indent(1), with manual corrections due to its many remaining
bugs.

No functional change.

make: use consistent indentation for statements and continuations

No binary change, except for line numbers in assertions in suff.c.

make: condense CondParser_ComparisonOp

No functional change.

make: fix memory leak when parsing '.if ${expr}' (since 2004-04-13)

$ (
echo 'VAR=value'
perl -e 'printf(".if \${VAR}\n.endif\n" x 5000);';
echo 'all:'
printf '\t%s\n' \
'@pid=$$$$; \' \
'ppid=$$(ps -o ppid -p "$$pid" | sed 1d); \' \
'ps -o vsz,rsz -p $$ppid | sed 1d'
) > leak-cond2.mk

$ make-2004.04.08.07.24.26 -r -f leak-cond.mk
2668
$ make-2004.04.13.16.06.23 -r -f leak-cond.mk
3964

$ echo $(((3964 - 2668) * 1000 / 5000))
259

This 259 is close enough to the 256 from Buf_Init in CondParser_Leaf.

tests/make: refine comments for parsing and evaluating conditions