Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r246296, r246298, r246346;

crunchide: support non-custom elf object layout.
crunchgen: Permit use of alternative linkers.
crunchide: Put e_shnum into a local variable. [1]

Submitted by: Pete Chou
Reviewed by: Christoph Mallon [1]

MFC r237574, r237625, r246256;

crunch: Sync some NetBSD changes.

crunchide:
Apr 11, 2009: fix some -Wsign-compare issues.
Sep 20, 1999: Free the right thing.

crunchgen:
Apr 14, 2009: Fix some WARNS=4 issues (-Wshadow -Wcast-qual)
Oct 30, 2004: Add (unsigned char) cast to ctype functions
Feb 5, 2001: fix nested extern.

examples:
Aug 30, 2007: NetBSD 36867 - trsp references are deprecated

In order to merge this I also had to merge some previous changes:
- Ensure crunchen uses the same make binary as the rest of the
build.
- Some amount of style(9): function definitions, header ordering,
and $FreeBSD$.

Copy head to stable/9 as part of 9.0-RELEASE release cycle.

Approved by: re (implicit)

Run with -B and just .POSIX.

Set .POSIX mode, along with disable parallel mode. Crunchgen parses the
output from make(1) and its picky what it should look like.
Also use make's conditional assignment operator rather than test and set.

Back out half of my previous change to support parallel makes.
The generated Makefile clashes with the src/bin/sh/Makefile, causing
it to try to use a rule to build something it doesn't need to.

Fix another parallel make problem with the generated make file.

Define the xxx_OBJPATHS earlier and then use it in the xxx_make
target because each obj is actually made through that.

This allows the crunch to work with -j32 on sun4v.

The makefile generated is still poor, though. It really shouldn't use
the general 'make all' to do the submakes in the app directories being
crunched because each of those objects is listed as a dependency in
the generated crunch makefile. Doing that really requires a unique rule
to generate them.

Add the SUBMAKE_TARGETS (like ppp_make) to the dependency list
for linking the crunched app so that when a parallel make is run
(like -j32 on sun4v), the link waits for the sub-make processes
to complete.

Commands like gmirror, graid3, ... and others which use dlopen() to load
classes from say, /lib/geom, cannot be statically linked completely.
Moreover, those shared objects may require other shared objects (i.e.
for geom, libraries like -lmd, -lcrypto).

The libs_so extension to crunchgen fixes this by allowing some libraries
to be linked in dynamically. This requires that a copy of rtld and the
shared libraries be made available to the crunched binary, and so is not
suitable for all environments. Crunchgen configurations which do not
use the 'libs_so' keyword are unaffected and produce identical binaries
with and without this commit.

Approved by: murray (mentor, in spirit), jhb
In collaboration with: Adrian Steinmann <ast at marabu dot ch>
MFC After: 6 weeks

In crunchgen(1), when calling make(1), don't redirect stderr to stdout,
just rely on the exit status to detect an error. This makes crunchgen(1)
safe to use with certain make(1) debugging flags.

MFC after: 1 week

Fix a 4.X -> 5.X upgrade problem: crunchgen used to emit a line 'MAKE=make'
or 'env MAKEOBJDIRPREFIX=... make' depending on the setting of
MAKEOBJDIRPREFIX in the environment. In any case this line kills the
original value of ${MAKE}. When during buildworld a new make is built (as
is the case during the upgrade) this causes a wrong make to be picked up
(the first one in the path). Use the same technique as Makefile.inc1:
create a MAKEENV variable and a CRUNCHMAKE that calls ${MAKE} with that
MAKEENV prefixed. Use CRUNCHMAKE instead of MAKE throughout the generated
makefile. This leaves the original ${MAKE} undisturbed.

The second chdir(1) that I trimmed is needed too
if the program has an object directory.

Explained by: Tim Kientzle <kientzle@acm.org>

chdir(1) into a source directory before attempting to determine the
OBJS list. This is needed to crunch any program that relies on the
correct .CURDIR setting, e.g. src/bin/csh.

Submitted by: Tim Kientzle <kientzle@acm.org>

Add a "special progname lib xxx ..." command to crunchgen
so the .lo files can be partially linked against libraries
which redefine symbols in the standard libs, or which reference
symbols in the objects.

Submitted by: Sam Leffler
MFC After: 3 days

Make source crunches work as well as object crunches; broken in the
last commit that fixed object crunches.

Repeat after me, "no excuses for not testing".

Make crunchgen crunch pre-built object files again, i.e.:

progs prog1
special prog1 objdir ../../prog1/obj
special prog1 objs prog1.o

This fixes a bug that I introduced around the time of 4.2-release.

Reported by: Larry Baird <lab@gta.com>

Use 'LIBS+= ...' instead of 'LIBS= ...' in the generated makefile
so that the user can override libraries at build time. This is
makes it easier to cross-compile crunch builds.

Suggested by: luigi
MFC after: 3 days

- Call `subclean' after `clean' for each program.
- Eliminate compiler warning.

`buildopts' may affect the selection of object files.
Make sure we pass $(BUILDOPTS) to the `clean' target
so that `make clean' works on the same set of object
files. Otherwise, we may end up with an incorrectly
built and up-to-date object file.

Liberal application of style(9).

Flag an error and remove the program from the list if there is
no source directory or no objects can be determined.

Initialise a structure variable.

Clean up the source directory finding code, and in the process fix a
problem that occurs when a program's source isn't found.

Use one of the TMPDIR env variable or _PATH_TMP instead of hardcoding
/tmp.

Submitted by: jedgar

Avoid potential problems with the existance of a shadow obj directory
confusing the 'make -f tmpfile' in the tmpfile, by creating it in /tmp.

Clarify an ambiguous 'else' introduced by a recent commit.

o check strdup() return values
o strcpy() -> strlcpy()
o sprintf() -> snprintf()
o mktemp() -> mkstemp()
o use err() instead of errx() in out_of_memory() function since
errno will probably be set

Submitted by: jedgar

Hopefully the last patch in a series to try and teach crunchgen to
handle MAKEOBJDIRPREFIX better, in conjunction with the -o and -p
flags.

Make crunchgen aware of obj directories. Formerly it had
/usr/obj hardwired in the code, now you can override it
with a command line option or MAKEOBJDIRPREFIX env. variable.
The above is useful to build picobsd-specific objects in some other
place than /usr/obj

While at it, fix documentation and change a few sprintf -> snprintf.

Add and document a new configuration command: buildopts.
This allows global make options to be specified within the crunch
configuration file.

Define a new special type: buildopts.

This allows build flags to be specified for a particular program from
within the crunch.conf file, eg:

prog ppp
special ppp buildopts -DNOKLDLOAD -DNOINET6 -DNONAT -DNOATM

This adds '-DNOKLDLOAD -DNOINET6 -DNONAT -DNOATM' to make targets
related to ppp when determining which object files to build and
when calculating dependencies and building the targets.

Cause prog_make targets to include a ``make obj'' rule if the -o flag
is given to crunchgen.
[ This fixes the previous commit which silently added ``make obj'' ]

A missing feature of crunchgen was the ability to supply make options on
a per program basis.

This has now been added in the following way:

* Harness the make header file that's specified with the -h argument:
- Allow the user to define $(OPTS) to specify make arguments that should
be added to every program target.
- Allow the user to define $(prog_OPTS) to specify make arguments that
should just be added to the build of 'prog'.

* Make sure that $(OPTS) and $(prog_OPTS) are defined when looking through
each program's make file to determine which object files to crunch.

* When building the crunchgen makefile add $(OPTS) and $(prog_OPTS)
to the depend and build rules for $(prog_OBJS).

Add a few features to crunchgen to simplify the use of existing
makefiles (for use with picobsd among other things).
See the manpage for details, but:

* -h makefile-include-name
can be used to specify a file to include in the makefiles
generated by crunchgen . This is a good place to specify make
variables such as RELEASE_CRUNCH, NOTHIS, NOTHAT and the like.

* special progname objvar variable_name
in the crunch config file declares a different variable than
OBJS to be used to get the list of objects.

* crunchgen now looks first for Makefile.<progname> in the current
directory to override the makefile in <progname> source dir.
This in many cases avoids the need to patch the original makefile
if the above two features are still not enough.

Approved-By: jordan

Assume ELF-style symbol names now.
Submitted by: jhay

Use /bin/pwd instead of a potentially shell builtin that does
not return the "real" path.
This makes it possible to have the .o files put in the /usr/obj
hierachy instead of in the /usr/src one.

Remove the necessity of -ldes and -lalias etc from release/Makefile.
When building a release, RELEASE_CRUNCH is defined for a `make' of
the objects required by the crunch of each program. The object list
is still obtained in the same way, so you must make sure that all
objects are built (empty if necessary) by this make. ppp/Makefile
provides an example.

Reviewed by: jkh

Add a new special called `keep' to the list of special options. It
provides for a means to specify an argument for crunchide's -k option.
(This is required by ntpdate.)

Submitted by: peter@rhiannon.clari.net.au (Peter Hawkins)

Use err(3).

Initialize memory obtained by malloc().

Detected by: phkmalloc -AJ

Enhance crunchgens understanding of our new /usr/obj/`pwd` strategy.

Make crunchgen match the new obj/ convention.

Remove trailing whitespace.

Add a '-l' option, which lists the argv[0]'s we will respond to. This will
simplify the src/release Makefile quite a bit.

This commit was generated by cvs2svn to compensate for changes in r1722,
which included commits to RCS files with non-trunk default branches.

This is the new crunch utility for making distribution floppies from
James da Silva. We need to integrate this before 1.1.5 so that we can
actually make the boot floppies.