Implement the ${<D} and ${<F} variables, as documented in the manual.
They are equivalent to ${<:H} and ${<:T} repectively, however the former
variants are more portable, as they are understood by GNU, FreeBSD and
NetBSD make too.

absence of those variables reported by Dario Niedermann <dnied at tiscali.it>
ok millert espie

Convert many atoi() calls to strtonum(), adding range checks and failure
handling along the way.
Reviews by Brendan MacDonell, Jeremy Devenport, florian, doug, millert

a bit more reallocarray (and kill ecalloc, which isn't used)
okay chl@

remove TIMESTAMP abstraction layer, prodded by theo.
while there, clean up includes.
use strtoll for ar timestamps (pretty much unused in reality, more
standard conforming than anything)

use idea from Todd to adapt to time_t being 32 bits OR 64 bits
(pedantically correct: INT_MIN would work just fine up to 1910 or so...)

okay millert@, gone thru a make build.

- SPECIAL_DEPRECATED -> SPECIAL_NOTHING
- nodes for .POSIX and .SCCS_GET (which don't do anything)
- zap remaining suffix crud.

new scaffolding:
- groupling list and HELDBACK state to avoid races in engine.
- parser recognizes lists of targets that shoul be grouped together
- OP_DOUBLE to mark nodes that have multiple lists of commands

make has had heuristics to avoid parallel make recursion.
Sometimes they mess up, so add .CHEAP/.EXPENSIVE to explicitly tell make
'hey this is not THAT bad' or 'worse than you think'.
agreed by guenther@, millert@ (and some tweaks)

Correct $OpenBSD$ stuff

two small changes:
- allow variables in SysV modifiers, as requested by matthieu@
(since recursive variables are an extension, this just extends the
extension)
- variation on :Q called :QL (quote list), which does quote every character
EXCEPT for whitespace. e.g.,

toto:
@for i in ${VAR:QL} ...

kill extra spaces at end of line

scaffolding to recognize special targets/sources using ohash functions.

use calloc() for the sake of regularity, prompted by deraadt@

Note that I *still* fill every pointer with explicit NULL, for portability.
filling with 0 bytes is not necessarily the same as NULL pointers.

A set of big related changes. okay millert@, tested further by kettenis@
and matthieu@

This all revolves around putting ALL global variables into one single big
hash, and using flags. This removes some impossible to understand stuff,
like old varfind, and allows for some nice stuff.
- each time we reference a global variable, we create it, possibly as a dummy
variable.
- each time we go to the environment, we remember it, thus we no longer go
back to it.

Lists of dependant changes:
- isolate changes to oldVars and checkEnvFirst.
- remove VAR_CMD and VAR_GLOBAL contexts. The only distinction is in parsevar.
Split Parse_DoVar into Parse_DoVar and Parse_CmdlineVar
- rework var modules around obtain_global_var, observe flags in various
functions like Var_Value and Var_Seti.
- Var_Seti/Var_Appendi are almost the same code, use that internally.
- add magic to handle the very special SHELL variable.
- introduce Var_Definedi for the cases where we don't want the actual
value, to simplify tests.
- add keyword .poison, parse it and set global flags accordingly.
- do poison_checks where needed.
- document poison.
- in for loops, set variable temporarily, so that Var_SubstVar will also
substitute it in varmodifiers expressions.

use stdint.h where appropriate. okay millert@

This is the first step in sanitizing the conditional parser.

Change the conditional recognition algorithm:
scan for a sequence of alphabetic characters, hash it, and compare it against
a small table (using ohash functions).
This makes Cond_Eval entry more logical, and allows for some shortcuts in
recognizing .include, .for, .undef.

This also means that conditionals must have an intervening blank between
the keyword and the actual test, e.g.,
.ifA
will no longer work.

(but no-one actually uses this, and it's highly obfuscated)

Okay miod@.

Mostly clean-up:
- cut up those huge include files into separate interfaces for all modules.
Put the interface documentation there, and not with the implementation.
- light-weight includes for needed concrete types (lst_t.h, timestamp_t.h).
- cut out some more logically separate parts: cmd_exec, varname, parsevar,
timestamp.
- put all error handling functions together, so that we will be able to
clean them up.
- more systematic naming: functioni to handle interval, function to handle
string.
- put the init/end code apart to minimize coupling.
- kill weird types like ReturnStatus and Boolean. Use standard bool (with a
fallback for non-iso systems)
- better interface documentation for lots of subsystems.

As a result, make compilation goes somewhat faster (5%, even considering
the largish BSD copyrights to read). The corresponding preprocessed
source goes down from 1,5M to 1M.

A few minor code changes as well: Parse_DoVar is no longer destructive.
Parse_IsVar functionality is folded into Parse_DoVar (as it knows what an
assignment is), a few more interval handling functions. Avoid calling
XXX_End when they do nothing, just #define XXX_End to nothing.

Parse_DoVar is slightly more general: it will handle compound assignments
as long as they make sense, e.g., VAR +!= cmd
will work. As a side effect, VAR++=value now triggers an error
(two + in assignment).
- this stuff doesn't occur in portable Makefiles.
- writing VAR++ = value or VAR+ +=value disambiguates it.
- this is a good thing, it uncovered a bug in bsd.port.mk.

Tested by naddy@. Okayed millert@. I'll handle the fallback if there is
any. This went through a full make build anyways, including isakmpd
(without mickey's custom binutils, as he didn't see fit to share it with me).

Synch with my current work.
Numerous changes:
- generate can build several tables
- style cleanup
- statistics code
- use variable names throughout (struct Name)
- recursive variables everywhere
- faster parser (pass buffer along instead of allocating multiple copies)
- correct parser. Handles comments everywhere, and ; correctly
- more string intervals
- simplified dir.c, less recursion.
- extended for loops
- sinclude()
- finished removing extra junk from Lst_*
- handles ${@D} and friends in a simpler way
- cleaned up and modular VarModifiers handling.
- recognizes some gnu Makefile usages and errors out about them.

Additionally, some extra functionality is defined by FEATURES. The set of
functionalities is currently hardcoded to OpenBSD defaults, but this may
include support for some NetBSD extensions, like ODE modifiers.

Backed by miod@ and millert@, who finally got sick of my endless patches...

Use the ohash_* that's now in libc.

This is the speed-up patch, which doubles make speed (almost).

Use the open hashing functions for global contexts instead of List in
var.c.

All the preliminary work to trim down local contexts means that we don't
suffer from the heavy initialization work that a hash table entails.

There is some make kludgery to:
- build the hashing functions as a library,
- recreate hashconsts.h, even if make depend was not invoked.

One point of the hashing scheme written was to separate the computation
of the hash function, and the hash lookup itself. This is very convenient
for make, because of those pesky special variables. hashconsts.h is there
to pre-hash the correct values, which replaces a few expensive string
comparisons with quick hash value comparisons, followed by one expensive
string comparison. The modulus MAGICSLOTS chosen in the Makefile is
ad-hoc: it is small enough to write a small switch without collision,
and will need changing if the hash function changes...

The function quick_lookup is the most important:
it either returns an index, for a local variable, or it does compute a
hashing value, and returns -1.

Another somewhat controversial decision is the use of string intervals.
This avoids either copying a string, or twiddling with a byte for cases
such as ${VAR}.

Finally, the variable name is stored within the variable itself. Since
a given variable name never changes, this makes sense. All that was needed
was a hash library with support for this. Note that the hashing table
holds only a variable pointer AND the corresponding hashing value, WITHOUT
a modulo hashtablesize. Two reasons:
- hash resizes can be done faster, without having to recompute hashing values.
- locality of access. The hash table fits into memory without problem. Once
a candidate slot is found, we check the complete hashing value. Probability
of a collision is very small (32 bits...). So bringing up the whole
variable in memory at once is good: the name will almost always match, in
which case we want the variable value as well, so it makes sense to put
them together.

The ohash functions implement open hashing, as described in Knuth, but with
a variable table size. Choosing powers of 2 sizes does not yield more
collisions, but it makes the hashing scheme much simpler. The thresholds at
which to expand/shrink the tables seem to work well in practice. The
default sizes were chosen such that the tables hardly ever shrink or expand
anyways (though I've tried with smaller/larger sizes to verify that the
shrinking/expanding worked correctly): larger Makefiles hold roughly
500/600 variables, which fits without trouble into a 1024-sized variable.

Disregard #ifdef STATS_HASH, this is some internal scaffolding I'm using
to measure make performance.

The only known issue with open-hashing is that deletions cannot create
empty slots, but do leave slots marked as `occupied once' so that lookup
works. We use a well-known optimization which records those pseudo-empty
slots while looking up values. If the value is not found, the pseudo-empty
slot is returned to be filled. If the value is found, it is swapped with
the pseudo-empty slot. This is an improvement in both cases, since this
shortens the length of lookup chains, eventually pushing the pseudo-empty
slots to the end.

Reviewed by millert@ and miod@