MFS r304472:

MFC r304006:

Avoid showing the bootstrap make command for check-old, etc.

Approved by: re (gjb)

MFS r304472:

MFC r304006:

Avoid showing the bootstrap make command for check-old, etc.

Approved by: re (gjb)

Copy stable/11@r303970 to releng/11.0 as part of the 11.0-RELEASE
cycle.

Prune svn:mergeinfo from the new branch, and rename it to RC1.

Update __FreeBSD_version.

Use the quarterly branch for the default FreeBSD.conf pkg(8) repo and
the dvd1.iso packages population.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r303272,r303273:

r303272:
SYSTEM_COMPILER: Rework the logic to allow a 'make test-system-compiler'.
r303273:
Fix empty WANT_COMPILER_TYPE when neither compiler is bootstrapped.

Approved by: re (kib)

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

WITH_META_MODE: Whitelist 'make kernel-toolchain'

Approved by: re (blanket, META_MODE)
Sponsored by: EMC / Isilon Storage Division

Fix top-level builds with fmake / 9.x.

Reported by: allanjude
Approved by: re (marius)
Sponsored by: EMC / Isilon Storage Division

WITH_META_MODE: Whitelist 'make kernel' and 'make world'.

installkernel is technically META_MODE safe but doesn't need an explicit
approval to use it since it's all disabled via .PHONY.

world uses 'make -B installworld' which already will disable META_MODE
via the -B (.MAKE.MODE=compat) usage.

Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division

tinderbox/universe: Only show arm64 warning if it was in the TARGETS list.

This was a flaw in my change in r287903 but also in the original change
in r282156 since it used empty(${TARGETS}) rather than empty(TARGETS).

Reported by: lidl
Approved by: re (gjb)
Sponsored by: EMC / Isilon Storage Division

Fix makeman showing dependency of DIRDEPS_BUILD->META_MODE.

This broke in r301887 with the meta mode whitelist. 'make showconfig'
still needs WITH_META_MODE support.

Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division

Fix build from stable/10 with fmake.

This was broken in r301888.

fmake does not look in share/mk by default and thus does not yet
have MK_META_MODE set with default.

Pointyhat to: bdrewery
Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division

WITH_META_MODE: Lessen the filemon(4) requirement scope.

- Move the sys.mk filemon requirement to bsd.init.mk as a warning.
This is intended only to show when building directly in a subdirectory
without filemon loaded.
- Move the error into Makefile and only apply it when building
from the META_TGT_WHITELIST target list.

-DNO_FILEMON can be used to suppress both the warning and the error but
makes WITH_META_MODE less useful. It will only compare build commands
in this mode rather than track all dependencies.

This fixes installing from a jail which doesn't need filemon in this
phase [1].

Reported by: Nikolai Lifanov <lifanov@mail.lifanov.com> [1]
Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division

WITH_META_MODE: Whitelist targets that are meta-mode-safe.

META_TGT_WHITELIST is added to define which build targets are safe for
meta mode. See comments for more details.

This fixes 'make delete-old-libs' to properly show the interactive
prompt.

Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division

Add more missing .PHONY

Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division

Only include bsd.compiler.mk for bmakes that use -m .../share/mk by default.

Otherwise they error when trying to include the file, which also has
its own dependencies.

Sponsored by: EMC / Isilon Storage Division

WITH_META_MODE: Enable bmake's missing meta rebuild feature

Fix bmake version upgrade logic to use the new bmake.

Otherwise it was only used on the next build.

This was a flaw in r295980.

MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division

Make armv6 hard float abi by default. Kill armv6hf.
Allow CPUTYPE=soft to build the current soft-float abi libraries.
Add UPDATING entry to announce this.

Approved by: re@ (gjb)

Follow-up r298220: Don't pass down META_MODE which will still enable it.

Reported by: Nikolai Lifanov <lifanov@mail.lifanov.com>
Sponsored by: EMC / Isilon Storage Division

META_MODE: Disable during installworld and similar.

META_MODE may create cookies during staging of files to WORLDTMP that would
also prevent installation of the files to the final DESTDIR, since the cookie
already exists. This is not yet the case but will be soon. Prevent other
similar issues by disabling META_MODE for any top-level install targets.

Sponsored by: EMC / Isilon Storage Division

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

Mark some more .PHONY targets.

Sponsored by: EMC / Isilon Storage Division

Define the *soft targets properly.

Sponsored by: EMC / Isilon Storage Division

Remove the old depend (mkdep) code and make FAST_DEPEND the one true way.

Reviewed by: emaste, hselasky (partial), brooks (brief)
Discussed on: arch@
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D5742

Use the newly minted Makefile.libcompat to implement libsoft libraries
for the armv6 ABI switch. This also make WITH_LIBSOFT functional on
the arm platform. As a transition thing, this seems to work even
without switching the ABI (we basically build the same libraries
twice when MK_LIBSOFT=yes until the ABI cut over next
month). MK_LIBSOFT remains default no.

Avoid MK_TESTS error on stable/10 by just preventing SUBDIR recursion.

Avoid bmake upgrade NO_MAN warning by just setting MAN to empty.

Suggested by: imp

Revert r296645 as it breaks stable/10->head builds.

FAST_DEPEND: Use .dinclude to enable full .depend logic in bmake.

The inclusion of .MAKE.DEPENDFILE (.depend) has special logic in make
to ignore stale/missing dependencies. bmake 20160220 added a '.dinclude'
directive that uses the special logic for .depend when including the file.

This fixes a build error when a file is moved or deleted that exists in a
.depend.OBJ file. This happened in r292782 when sha512c.c "moved" and an
incremental build of lib/libmd would fail with:
make: don't know how to make /usr/src/lib/libcrypt/../libmd/sha512c.c. Stop

Now this will just be seen as a stale dependency and cause a rebuild:
make: /usr/obj/usr/src/lib/libmd/.depend.sha512c.o, 13: ignoring stale .depend for /usr/src/lib/libcrypt/../libmd/sha512c.c
--- sha512c.o ---
...
This rebuild will only be done once since the .depend.sha512c.o will
be updated on the build with the -MF flags.

This also removes -MP being passed for the .depend.OBJ generation (which
would create fake targets for system headers) since the logic is no
longer needed to protect from missing files.

Sponsored by: EMC / Isilon Storage Division

Fix bmake upgrade NO_MAN warnings.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Fix make -n upgrade_checks.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Don't ever create object directories here with MK_AUTO_OBJ.

Sponsored by: EMC / Isilon Storage Division

Add order for installworld/installkernel.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Support a WANT_MAKE_VERSION.

This will be used soon for .dinclude support in FAST_DEPEND.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Fix make universe when running with non POSIX/C locales using a locale sensitive
pattern

Reported by: many
Submitted by: jilles

Simplify running the FreeBSD test suite

Replace `make regress` (legacy test make target) and `make test` (incomplete
test make target added with the FreeBSD test suite) with make check as it's
consistent with other open source projects.

`make check` defaults to running tests from `.OBJDIR`, but can be overridden
with the `CHECKDIR` variable.

Add `make checkworld` target to simplify running the FreeBSD test suite from
`TESTSBASE` (i.e. the top-level tests directory), similar to buildworld.

Document `make check` and `make checkworld` in build(7).

Other minor changes:

- Rename intermediate file (`Kyuafile.auto`) to `Kyuafile` to simplify
`make check`.
- Remove terse warnings attached to `beforetest`/`aftertest`.
- Add kyua binary check to check target in suite.test.mk; error out if it's
not found

The MFC is [partly] contingent on other build related changes being MFCed.

Differential Revision: https://reviews.freebsd.org/D4406
MFC after: 2 months
X-MFC to: stable/10
Relnotes: yes
Reviewed by: bdrewery, Evan Cramer <eccramer@gmail.com>
Sponsored by: EMC / Isilon Storage Division

Add order for installworld/distribution.

Sponsored by: EMC / Isilon Storage Division

Add make universe targets "kernels" and "worlds".

"make kernels" is now shorthand for "make universe -DMAKE_JUST_KERNELS"
"make worlds" is now shorthand for "make universe -DMAKE_JUST_WORLDS"

The kernels target includes modules (unless you add -DNO_MODULES).

And of course you can still add all the other universe options, such as
"make kernels TARGETS=arm" to build kernels for all arm arches, or
TARGET_ARCH=armv6 to build all armv6 kernels, etc.

Reviewed by: imp

Always try to upgrade to bmake if not already using it.

This is mostly targetting stable/10 which requires bmake to build and
has issues upgrading from <10.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
PR: 198062

Revert r293286. It was not intended to come in yet.

Move the MAKEOBJDIRPREFIX value guard to sys.mk and expand to MAKEOBJDIR.

This will ensure that the variable was not set as a make override, in
make.conf, src.conf or src-env.conf. It allows setting the value in
src-env.conf when using WITH_AUTO_OBJ since that case properly handles
changing .OBJDIR (except if MAKEOBJDIRPREFIX does not yet exist which is
being discussed to be changed).

This change allows setting a default MAKEOBJDIRPREFIX via local.sys.env.mk.

Sponsored by: EMC / Isilon Storage Division

Update advice on obj disk space requirements

The Makefile is not the best place for this information, but at least it
is now less out of date.

Start support for the RISC-V 64-bit architecture developed by UC Berkeley.

RISC-V is a new ISA designed to support computer research and education, and
is now become a standard open architecture for industry implementations.

This is a minimal set of changes required to run 'make kernel-toolchain'
using external (GNU) toolchain.

The FreeBSD/RISC-V project home: https://wiki.freebsd.org/riscv.

Reviewed by: andrew, bdrewery, emaste, imp
Sponsored by: DARPA, AFRL
Sponsored by: HEIF5
Differential Revision: https://reviews.freebsd.org/D4445

Fix errors being ignored in many phases of the build since the bmake integration.

Say it with me, "I will not chain commands with && in Makefiles"

This was originally fixed and explained quite well by bde@ in r36074. The
initial bmake integration caused 'set -e' to stop being used which lead to
r252419. Later 'set -e' expectations were fixed with bmake in r254980.

Because of the && here, errors would be ignored when building in parallel and
a dependency failed. Such as bootstrap-tools since it builds everything in
parallel. If any tool failed in obj/depend/all, it would just ignore the error
and continue to build. This later would result in cascaded errors that only
confused the real issue. This could also cause commands after the failed
command to still execute, leading to more confusion.

This should be fine if the command is in a sub-shell such as: (cmd1 && cmd2)

This reverts r252419.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Rename META_MODE option to DIRDEPS_BUILD

This allows META_FILES option to be renamed META_MODE.
Also add META_COOKIE_TOUCH for use in targets that can benefit
from a cookie when in meta mode.

Differential Revision: https://reviews.freebsd.org/D4153
Reviewed by: bdrewery

Move 'make cleandir' from Makefile to Makefile.inc1.

This leads the way for fixing cross-build cleanup, and eventually replacing
'cleandir' with it during the build.

Sponsored by: EMC / Isilon Storage Division
MFC after: 2 weeks

Add built-in ccache build support via WITH_CCACHE_BUILD option.

ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not
used to achieve a safe pseudo-incremental build. This is explained in
more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the
cache, but with a full cache saves 30-50% in build times. When combined with
the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the
resulting dependency file, which it does not do when using mkdep(1)/'CC
-E'. Stats are provided at the end of this message.

This removes the need to modify /etc/make.conf with the CC:= and CXX:=
lines which conflicted with external compiler support [3] (causing the
bootstrap compiler to not be built which lead to obscure failures [4]),
incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid
Clang errors with parenthesis, and did not work with META_MODE.

The option name was picked to match the existing option in ports. This
feature is available for both in-src and out-of-src builds that use
/usr/share/mk.

Linking, assembly compiles, and pre-processing avoid using ccache since it is
only overhead. ccache does nothing special in these modes, although there is
no harm in calling it for them.

CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap
compiler to hash the content of the compiler binary to determine if it
should be a cache miss. For external compilers the 'mtime' option is used
as it is more efficient and likely to be correct. Future work may optimize the
'content' check using the same checks as whether a bootstrap compiler is needed
to be built.

The CCACHE_CPP2 pessimization is currently default in our devel/ccache
port due to Clang requiring it. Clang's -Wparentheses-equality,
-Wtautological-compare, and -Wself-assign warnings do not mix well with
compiling already-pre-processed code that may have expanded macros that
trigger the warnings. GCC has so far not had this issue so it is allowed to
disable the CCACHE_CPP2 default in our port.

Sharing a cache between multiple checkouts, or systems, is explained in
the ccache manual. Sharing a cache over NFS would likely not be worth
it, but syncing cache directories between systems may be useful for an
organization. There is also a memcached backend available [5]. Due to using
an object directory outside of the source directory though you will need to
ensure that both are in the same prefix and all users use the same layout. A
possible working layout is as follows:
Source: /some/prefix/src1
Source: /some/prefix/src2
Source: /some/prefix/src3
Objdir: /some/prefix/obj
Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj'
This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace
all absolute paths to be relative. Using something like this is required due
to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache
adds into its hash for the object without CCACHE_BASEDIR.

distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc.
I have not personally tested this and assume it will not mix well with
using the bootstrap compiler.

The cache from buildworld can be reused in a subdir by first running
'make buildenv' (from r290424).

Note that the cache is currently different depending on whether -j is
used or not due to ccache enabling -fdiagnostics-color automatically if
stderr is a TTY, which bmake only does if not using -j.

The system I used for testing was:
WITNESS
Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes
DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
The arc was fully populated with src tree files and ccache objects.
RAM: 76GiB
CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16

The WITH_FAST_DEPEND feature was used for comparison here as well to show
the dramatic time savings with a full cache.

buildworld:
x buildworld-before
+ buildworld-ccache-empty
* buildworld-ccache-full
% buildworld-ccache-full-fastdep
# buildworld-fastdep
+-------------------------------------------------------------------------------+
|% * # +|
|% * # +|
|% * # xxx +|
| |A |
| A|
| A |
|A |
| A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 3744.13 3794.31 3752.25 3763.5633 26.935139
+ 3 4519 4525.04 4520.73 4521.59 3.1104823
Difference at 95.0% confidence
758.027 +/- 43.4565
20.1412% +/- 1.15466%
(Student's t, pooled s = 19.1726)
* 3 1823.08 1827.2 1825.62 1825.3 2.0785572
Difference at 95.0% confidence
-1938.26 +/- 43.298
-51.5007% +/- 1.15045%
(Student's t, pooled s = 19.1026)
% 3 1266.96 1279.37 1270.47 1272.2667 6.3971113
Difference at 95.0% confidence
-2491.3 +/- 44.3704
-66.1952% +/- 1.17895%
(Student's t, pooled s = 19.5758)
# 3 3153.34 3155.16 3154.2 3154.2333 0.91045776
Difference at 95.0% confidence
-609.33 +/- 43.1943
-16.1902% +/- 1.1477%
(Student's t, pooled s = 19.0569)

buildkernel:
x buildkernel-before
+ buildkernel-ccache-empty
* buildkernel-ccache-empty-fastdep
% buildkernel-ccache-full
# buildkernel-ccache-full-fastdep
@ buildkernel-fastdep
+-------------------------------------------------------------------------------+
|# @ % * |
|# @ % * x + |
|# @ % * xx ++|
| MA |
| MA|
| A |
| A |
|A |
| A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 571.57 573.94 571.79 572.43333 1.3094401
+ 3 727.97 731.91 728.06 729.31333 2.2492295
Difference at 95.0% confidence
156.88 +/- 4.17129
27.4058% +/- 0.728695%
(Student's t, pooled s = 1.84034)
* 3 527.1 528.29 528.08 527.82333 0.63516402
Difference at 95.0% confidence
-44.61 +/- 2.33254
-7.79305% +/- 0.407478%
(Student's t, pooled s = 1.02909)
% 3 400.4 401.05 400.62 400.69 0.3306055
Difference at 95.0% confidence
-171.743 +/- 2.16453
-30.0023% +/- 0.378128%
(Student's t, pooled s = 0.954969)
# 3 201.94 203.34 202.28 202.52 0.73020545
Difference at 95.0% confidence
-369.913 +/- 2.40293
-64.6212% +/- 0.419774%
(Student's t, pooled s = 1.06015)
@ 3 369.12 370.57 369.3 369.66333 0.79033748
Difference at 95.0% confidence
-202.77 +/- 2.45131
-35.4225% +/- 0.428227%
(Student's t, pooled s = 1.0815)

[1] https://ccache.samba.org/performance.html
[2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html
[3] https://reviews.freebsd.org/D3484
[5] https://github.com/jrosdahl/ccache/pull/30

PR: 182944 [4]
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
Relnotes: yes

Rework the 'make -n -n' feature such that '-n' recurses and '-N' does not.

Bmake has a documented feature of '-N' to skip executing commands which is
specifically intended for debugging top-level builds and not recursing into
sub-directories. This matches the older 'make -n' behavior we added which made
'-n -n' the recursing target and '-n' a non-recursing target.

Removing the '-n -n' feature allows the build to work as documented in
the bmake manpage with '-n' and '-N'. The older '-n -n' feature was also
not documented anywhere that I could see.

Note that the ${_+_} var is still needed as currently bmake incorrectly
executes '+' commands when '-N' is specified.

The '-n' and '-n -n' features were broken for several reasons prior to this.
r251748 made '_+_' never expand with '-n -n' which resulted in many
sub-directories not being visited until fixed 2 years later in r288391, and
many targets were given .MAKE over the past few years which resulted in
non-sub-make commands, such as rm and ln and mtree, to be executed.

This should also allow removing some indirection hacks in bsd.subdir.mk and
other cases of .USE that have a .MAKE by using '+'.

Sponsored by: EMC / Isilon Storage Division
Discussed on: arch@ (mostly silence)

Tweak the guard more to suggest 'all' if SUBDIR_OVERRIDE is specified. In that
case 'all' does make sense.

MFC after: 2 weeks
X-MFC-With: r289411
Sponsored by: EMC / Isilon Storage Division

Tweak the default target to not suggest 'all' since it really doesn't do
anything useful for most users.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Consider top-level targets to be .PHONY as bmake won't build them
otherwise if a file with the same name is found in the directory.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

Add installconfig as a top level target

Fix the .MAKE added in r251750 to properly support the historical -n -n.

The condition used matches the condition in sys.mk for setting _+_ to blank
or +.

With this -n will continue to not descend into Makefile.inc1, while -n -n will
and cause Makefile.inc1's target to run with -n.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Rework r287900 to keep arm64/aarch64 stable in the TARGETS list.

This is relevant for makeman using the 'make targets' output in src.conf(5).

This makes a _UNIVERSE_TARGETS that removes arm64 if the build
requirements are not met.

Get arm64/aarch64 into 'make targets' output to fix makeman.

Fix generation of src.conf.5

Since makeman turns all options on, we need to guard somethings from
make(showconfig)

targets now needs .PHONY

Move the universe done dependency on the worlds completing out of the
loop, we only need to add it once.

Obtained from: ABT Systems Ltd

Remove unneeded dependencies, these are now handled through the _done
target.

Obtained from: ABT Systems Ltd

Add a new target universe_${target}_done to print the completion message.
Without this we could print this message in the wrong place when building
with MAKE_JUST_WORLDS is set.

Obtained from: ABT Systems Ltd

Add a universe_${target}_worlds target to simplify the logic to find when
to start building kernels.

Obtained from: ABT Systems Ltd

A more compatible fix to MK_META_MODE not being defined. Also, encase
bmake specific constructs not needed for make bootstrap so fmake
doesn't see them. This works with fmake just well enough for us to
build bmake to build the rest of the tree without fatal errors. Tested
only with fmake package.

Add a comment to the end of the world and kernel cases in the universe
target to help follow the make magic.

Obtained from: ABT Systems Ltd

If MK_META_MODE is unset, assume a value of no.

This is needed to build HEAD on FreeBSD 10.1,
which has bmake, but does not have
/usr/share/mk/src.opts.mk

Reviewed by: sjg
Reported by: jenkins

Add META_MODE support.

Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision: D2796
Reviewed by: brooks imp

Don't add arm64 to universe builds if the user provided a TARGETS list

Differential Revision: https://reviews.freebsd.org/D2375
Reported by: andrew
Reviewed by: andrew, imp
Sponsored by: The FreeBSD Foundation

Fix a typo (linnker -> linker)

Add arm64 to universe if binutils is available.

arm64 relies on an external binutils port or package right now, because
the in-tree linker from binutils 2.17.50 does not support arm64. Add
arm64 to universe if the linker is available. If not output a message
that arm64 is skipped.

buildworld and buildkernel use the external binutils automatically, so
it's sufficient to run 'pkg install aarch64-binutils' to build
FreeBSD/arm64.

Differential Revision: https://reviews.freebsd.org/D2302
Reviewed by: andrew, imp
Sponsored by: The FreeBSD Foundation

Make TARGET_ARCH=powerpc64 work without TARGET=powerpc.

Start to import support for the AArch64 architecture from ARM. This change
only adds support for kernel-toolchain, however it is expected further
changes to add kernel and userland support will be committed as they are
reviewed.

As our copy of binutils is too old the devel/aarch64-binutils port needs
to be installed to pull in a linker.

To build either TARGET needs to be set to arm64, or TARGET_ARCH set to
aarch64. The latter is set so uname -p will return aarch64 as existing
third party software expects this.

Differential Revision: https://reviews.freebsd.org/D2005
Relnotes: Yes
Sponsored by: The FreeBSD Foundation

Revert r271159, Mis-patched the tree.

Pointed out by: kib

The USB LED driver for the Dream Cheeky WebMail Notifier.

Reviewed by: hselasky

Create the native-xtools target. This target creates only the cross
building toolchain for the host computer. This toolchain produces
TARGET_ARCH and assumes the rest of the system contains libraries for
the target. It is intended to be used in a "qemu-user jail" where all
the binaries would otherwise be the target architecture's to build
ports. However, emulation of the compilers is too slow, so we build
native binaries for that. Rather than use the xdev produced binaries,
with all their weird links and paths, these binaries use the native
paths. They will not work unless installed into the qemu-user jail.

Differential Revision: https://phabric.freebsd.org/D518
Reviewed by: sbruno@

Add compat shims for XDEV and XDEV_ARCH so we don't break all the
users of them. Also, add a note to updating. We'll keep these shims at
least until the 12 branch.

Explicitly disable the build of tests when building bmake.

During "make buildworld", building bmake is (one of) the very first steps
and we should not be building any of its tests. Conceptually, this is the
right thing to do 1) for build simplicity reasons and 2) because there is
no need to build any tests this early on.

In practice, this fixes tinderbox builds of CURRENT from 9.x when MK_TESTS
is enabled. This is because bsd.test.mk needs some modern bmake features
not present in 9.x (:tW) and tinderbox is forcing the build to use the
CURRENT share/mk files from the very beginning (see r266617). By skipping
the build of the tests when still using the host make, we omit the problem.
Arguably, what tinderbox is doing is wrong and needs to be addressed, but
that is a separate issue.

Separate out the links creation from the other targets. This was
supposed to have been done for the original commit, but somebody
forgot.

Pointy-hat-to: imp@

Remove ia64 from the list of known architectures and add an entry to
UPDATING. This is the first step towards the removal of ia64 from
head. A buildworld for ia64 will now yield:

% make buildworld
make[1]: "/usr/src/Makefile.inc1" line 151: Unknown target ia64:ia64.

While here, trim the ia64-specific additions from ObsoleteFiles.inc

Discussed at: BSDcan

grep -L returns non-zero status if none of the files had the pattern
in them. This is often the case, so just ignore the return
code. Actual errors that are found will also be detected downstream in
the rare cases where the return code is 2 instead of 1.

We haven't done anything with _UPGRADING in ~forever (was present, but
not needed, in FreeBSD 6.x, and has been absent in newer versions).
This was needed to upgrade from 3.x -> 4.x, once upon a time.

Remove a few more vestiges of allowing WITHOUT_BMAKE to imply you want
to buid with fmake.

No need to install man pages for bootstrapping new make. Some
environments (that I can't reproduce locally, but that others have
reported) seem to get tripped up by this man page install. There's
really no need to do it, so turn off the man pages using the most
portable method. We can't just directly set MK_MAN=no here because
we're bootstrapping in the host environment and such a setting was
forbidden until very recently. NO_MAN= can produce a warning, but for
now the warning is benign.

Remove support for WITHOUT_BMAKE. bmake is now the only make that can
build world, so it is the only make we build or install. fmake is
still in the tree, but disconnected, and upgrades from older systems
that still have bmake has not been removed, but its state has not been
tested (it should work given how minimal the work to upgrade to bmake
is).

Omit from the universe build all config files tagged with
#NO_UNIVERSE. Many of these config files are important examples, but
add little to no regresive value to the intended purpose of
UNIVERSE. We now build over 120 kernels during universe. There's
really little to no value to this over building say 60 or even 30 of
them (either is still a way too big number). This is especially true
for kernels that are nothing more than including a common base and
adding a static DTB file. Start by pruning 1/3 of the arm kernels that
add little regresion value.

Merge from CheriBSD:

commit 1d1b908107255ffdff4d17f015d8f057d73cc6cb
Author: Brooks Davis <brooks@one-eyed-alien.net>
Date: Fri Mar 28 16:24:45 2014 +0000

Add a long needed seatbelt.

Exit with an error when make is called without a target at the top level
rather than poluting the source tree and causing use confusion in future
builds.

commit a9d9aa341b2f4308a227ab460ba85f1f287ad028
Author: Brooks Davis <brooks@one-eyed-alien.net>
Date: Tue Apr 29 16:06:12 2014 +0000

Simplify seatbelt added in 1d1b908 based in feedback.

Discussed with: imp@FreeBSD.org

Reviewed by: imp
Sponsored by: DARPA, AFRL

Prevent building tests when bootstrapping make.

Should fix the breakage reported by tinderbox when WITHOUT_BMAKE is set,
which was probably introduced in r263346..

Document the xdev* targets.

PR: 188519
Submitted by: Idwer Vollering <vidwer at gmail.com>
Reviewed by: bapt
MFC after: 1 week

We no longer support upgrading from FreeBSD 4, so we don't need the
NOMAN and NOSHARED defines here. They have been obsolete for almost a
decade anyway.

Remove support for legacy mips*eb names. Remove tests for
TARGET_BIG_ENDIAN.

Add a new ARM TARGET_ARCH, armv6hf. This is considered experimental.

This targets the existing ARMv6 and ARMv7 SoCs that contain a VFP unit.
This is an optional coprocessors may not be present in all devices, however
it appears to be in all current SoCs we support.

armv6hf targets the VFP variant of the ARM EABI and our copy of gcc is too
old to support this. Because of this there are a number of WITH/WITHOUT
options that are unsupported and must be left as the default value. The
options and their required value are:
* WITH_ARM_EABI
* WITHOUT_GCC
* WITHOUT_GNUCXX

In addition, without an external toolchain, the following need to be left
as their default:
* WITH_CLANG
* WITH_CLANG_IS_CC

As there is a different method of passing float and double values to
functions the ABI is incompatible with existing armv6 binaries. To use
this a full rebuild of world is required. Because no floating point values
are passed into the kernel an armv6 kernel with VFP enabled will work with
an armv6hf userland and vice versa.

Remove the armv6eb architecture as it is unused, and almost certainly
broken. None of our kernels can boot armv6eb. The little-endian kernels do
not have the required code to be able to switch endian when running a
big-endian executable.

Approved by: re (gjb)

Further refinement to bmake bootstrapping

Include PROGNAME and DESTDIR in ${MMAKE} so that it doesn't need to be
passed to each make invocation.

Suggested by: hrs
Reviewed by: hrs
Approved by: re (gjb)

Fix make bootstrapping with WITH_DEBUG_FILES=

A HEAD buildworld on 9.x first bootstraps bmake, but this failed when
building with standalone debug. Pass in the PROGNAME override to the
'make all' stage as well as 'make install' so that the .debug file is
created with the correct name.

Reviewed by: sjg
Sponsored by: The FreeBSD Foundation
Approved by: re (rodrigc)

During universe/tinderbox export MAKE_JOB_ERROR_TOKEN=no
This avoids aborting everything when one kernel fails.

Reviewed by: obrien

Don't let user specified DESTDIR, break building our chosen make.

When we need to build using the in-tree make,
switch at the earliest opportunity.
In the case of fmake vs bmake, this helps ensure correct load handling.

Reviewed by: obrien

Fix bug in universe where if upgrade_checks wants a new make,
it gets built 16 times in parallel in the same location.
While we are at it, until we finish getting rid of fmake,
be explicit about the make we want to use, thus avoid the problem
of the temp make being the wrong version.

Reviewed by: obrien

Correct typo specifying jflags.

Document tip on how to build all kernels quickly.

Flag recursive make targets with .MAKE
so that job token pipe is passed to them.
To avoid surprising anyone, only add .MAKE to ${TGTS} when -n
has not been specified (at least for Makefile).

Reviewed by: obrien

bmake does not like universe_${target}_prologue in the .ORDER
when there is an explicit dependency for it anyway.
Remove it so both bmake and fmake do the same thing.

Reviewed by: OBrien

Make the instruction order for buildworld agree with UPDATING and the
Handbook.

Reviewed by: imp, bdrewery, netchild
Approved by: gjb (mentor)

Reverse the sense of the test wrt bmake, and guard against
MK_BMAKE not being defined.

If doing buildworld -DWITH_BMAKE make sure bmake does the build.

Ensure make -j N universe works correctly, by checking for an up-to-date
make before starting the universe targets themselves. Otherwise, all of
the targets would attempt to build make simultaneously, overwriting each
other's copies of the make object files and executable. This could lead
to strange errors, for example when partially-written make executables
are invoked.

Also amend r216620, to make the rest of universe wait properly until the
upgrade_checks target is finished, by adding universe_${target}_prologue
to the .ORDER target. Otherwise, make will be too smart for its own
good, and start building the universe targets simultaneously with the
prologues anyway.

MFC after: 1 week

When building universe ensure the required worlds are finished before
starting the kernels. Before this the kernels would be built as part of the
last architecture universe target. There can cause problems when this world
finishes before the other worlds as the host compiler may be picked up
rather than the target compiler.

The solution is to add a target to build the universe kernels that depends
on all the world targets finishing. As we may not be building a world only
depend on it when MAKE_JUST_KERNELS is undefined.

Add the buildLINT convenience target, handy for build automation.
The automation can set TARGET_ARCH and TARGET and then make various
top-level targets, including buildLINT and buildkernel (with
KERNCONF=LINT). Previously there was no way to generate the LINT
kernel configuration without having to do something exceptionally
painful.

Add support for bmake. This includes:
1. Don't do upgrade_checks when using bmake. As long as we have WITH_BMAKE,
there's a bootstrap complication in ths respect. Avoid it. Make the
necessary changes to have upgrade_checks work wth bmake anyway.
2. Remove the use of -E. It's not needed in our build because we use ?= for
the respective variables, which means that we'll take the environment
value (if any) anyway.
3. Properly declare phony targets as phony as bmake is a lot smarter (and
thus agressive) about build avoidance.
4. Make sure CLEANFILES is complete and use it on .NOPATH. bmake is a lot
smarter about build avoidance and should not find files we generate in
the source tree. We should not have files in the repository we want to
generate, but this is an easier way to cross this hurdle.
5. Have behavior under bmake the same as it is under make with respect to
halting when sub-commands fail. Add "set -e" to compound commands so
that bmake is informed when sub-commands fail.
6. Make sure crunchgen uses the same make as the rest of the build. This
is important when the make utility isn't called make (but bmake for
example).
7. While here, add support for using MAKEOBJDIR to set the object tree
location. It's the second alternative bmake looks for when determining
the actual object directory (= .OBJDIR).

Submitted by: Simon Gerraty <sjg@juniper.net>
Submitted by: John Van Horne <jvanhorne@juniper.net>

Not all Pmake derivatives silently handle empty shell output, so ensure there
is something for make(1) to consume. Bmake gives output such as:
"warning: Couldn't read shell's output for "/bin/sh -c true"
Note we parted from traditional Pmake behavior in r18864 / r18255.

Merging of projects/armv6, part 3

r238211:
Support TARGET_ARCH=armv6 and TARGET_ARCH=armv6eb

This adds a new TARGET_ARCH for building on ARM
processors that support the ARMv6K multiprocessor
extensions. In particular, these processors have
better support for TLS and mutex operations.

This mostly touches a lot of Makefiles to extend
existing patterns for inferring CPUARCH from ARCH.
It also configures:
* GCC to default to arm1176jz-s
* GCC to predefine __FreeBSD_ARCH_armv6__
* gas to default to ARM_ARCH_V6K
* uname -p to return 'armv6'
* make so that MACHINE_ARCH defaults to 'armv6'
It also changes a number of headers to use
the compiler __ARM_ARCH_XXX__ macros to configure
processor-specific support routines.

Submitted by: Tim Kientzle <kientzle@freebsd.org>

Add "hier" as an alternate spelling of "hierarchy" to match hier(9).

Assume a big-endian default on MIPS and drop the "eb" suffix from MACHINE_ARCH.
This makes our naming scheme more closely match other systems and the
expectations of much third-party software. MIPS builds which are little-endian
should require and exhibit no changes. Big-endian TARGET_ARCHes must be
changed:
From: To:
mipseb mips
mipsn32eb mipsn32
mips64eb mips64

An entry has been added to UPDATING and some foot-shooting protection (complete
with warnings which should become errors in the near future) to the top-level
base system Makefile.

Add a kernel-toolchains target like the toolchains target, but building only
the parts of the toolchain necessary to build kernels.

delete-old does not have delete-old-libs dependency.
Reflect this in the comment.

PR: conf/163993
Submitted by: Eugen Konkov <kes-kes at yandex ru>
MFC after: 3 days

Replace 'chflags' with 'file flags' in a comment, since 'chflags'
is a command, not a flag itself.

While here, wrap a line for readability.

Submitted by: arundel
MFC after: 3 days
X-MFC-With: 227769, 227770

Fix the note in r227769 to be less specific to the immutable flag.

Submitted by: arundel
MFC after: 3 days
X-MFC-With: r227769

Add a note to src/Makefile that explains that 'rm' runs twice because the
second invocation only needs to operate on files with the immutable flag
set.

Submitted by: arundel (via private email) (original version)
Discussed on: -toolchain@
MFC after: 3 days

It's a bit odd, but "make update" in src/ can also update the ports/,
doc/, and now www/ trees, but only using the "cvsup" transport.

When "make update" is run using a tree's makefile, it can also use
"cvs" (except for www/) and "svn" (only src/).

Clean up documentation and code regarding "make update":

- Increase oddness by adding support for WWWSUPFILE and NO_WWWUPDATE to
Makefile.inc1 (analogous to PORTSSUPFILE/NO_PORTSUPDATE and
DOCSUPFILE/NO_DOCUPDATE; WWWSUPFILE already supported by www/Makefile).

- Document all trees that support CVS_UPDATE.

- Document all trees that support SUP_UPDATE.

- Document SVN_UPDATE.

- Document NO_WWWUPDATE.

- make.conf(5) mistakenly said that *SUPFILE* had defaults.

- Add an example entry for WWWSUPFILE.

Disconnect sun4v architecture from the three.

Some files keep the SUN4V tags as a code reference, for the future,
if any rewamped sun4v support wants to be added again.

Reviewed by: marius
Tested by: sbruno
Approved by: re

Add the mipsn32eb TARGET_ARCH to the list of mips targets for the
XLRN32 kernel config.

Reviewed by: imp, jmallet

If building (custom) FreeBSD images people tend to patch param.h. In case
this happens just before the build is started (within the same second)
CHECK_TIME actually triggers thinking param.h is in the future (see f_Xtime,
c_Xtime logi in find(1) sources for the details in !F_EXACTTIME case).
Using the -mtime -0s (seconds, rather than no unit) avoids this 1s race.

Submitted by: ed (2009-05-03)
Reviewed by: cperciva (2009-05-03), emaste
Tested by: bz (for almost two years)
MFC after: 4 days

Update recommendation for mergemaster. The -a and -i options are exclusive.

PR: docs/156271
Submitted by: Fedor Dikarev <fedor.dikarev@gmail.com>

Improve the distributeworld target in Makefile.inc1 and update the release
infrastructure to use it. make distributeworld can now be used without
preparing its environment first and installs games into its distribution
using the regular make distribute logic instead of post-processing with
a script.

Also add two new targets, packageworld and packagekernel, that tar up the
results of distributeworld and distributekernel (also new), respectively.

Fully honor KERNSRCDIR for 'make universe' if it is set.

MFC after: 1 week

- Add a new UNIVERSE_TARGET variable for 'make universe'. If it is set,
then that target is invoked for each architecture rather than the
default action of building world and kernels for each architecture.
- Add a 'make toolchains' wrapper which uses UNIVERSE_TARGET to build
toolchains for all architectures.
- Document JFLAG, MAKE_JUST_KERNELS, and MAKE_JUST_WORLDS variables for
'make universe'.

Reviewed by: bz
MFC after: 1 week

Setting TARGET and TARGET_ARCH needs to be done in _MAKE, not in the
TGTS rule as _MAKE is used elsewhere. This should fix make world.

Move the architecture guessing from Makefile.inc1 to Makefile. We
need to do this because variables specified on the command line
override those specified in the Makefile. This is why we also moved
from TARGET to _TARGET in Makefile, and then set TARGET on the command
line when we fork a submake with Makefile.inc1.

This makes mips/mips work again, even without the workaround committed to
lib/libc/Makefile.

Back out r217734. [1]

Properly document what `make targets` is supposed to list to avoid
further confusion given the place the target sits. Should have happened
with r217125.

Requested by: imp [1]
Reviewed by: rwatson

For `make tinderbox` there is no need to print the extra commands.

MFC after: 5 days

Make `make tinderbox` work with MAKEOBJDIRPREFIX set (or in possibly other
combinations) by forcing FAILFILE into .CURDIR as we do for all other
universe output files. [1] Similarly make FAILFILE start with "_." as well.

Reviewed by: silence-on-src [1]
MFC after: 6 days

In `make targets` print the make variable TARGETS as we expect it rather
than the grammatically better sounding variant without the 'S'. This
allows copy and paste and is less confusing.

Properly document what the top-level `make tinderbox` does.

Reviewed by: jmallett
MFC After: 3 days
X-MFC: build.7 only

Provide up-to-date estimates that are pessimal today...

Proving once again, I can't typo: fix a type.

Minor improvements in the wording of a comment. Document tinderbox target.

make targets

This produces a list of currently supported targets. Here "supported"
means "built in make universe" on the theory that those targets are
more supported than any that might work in 'make buildworld TARGET=x
TARGET_ARCH=y' since the latter are less tested.

Suggested by: rwatson

Retire TARGET_ABI.

Implement MACHINE_ARCH=mips64e[lb] to build N64 images. This replaces
MACHINE_ARCH=mipse[lb] TARGET_ABI=n64.

MACHINE_ARCH=mipsn32e[lb] has been added, but currently requires
WITHOUT_CDDL due to atomic issues in libzfs. I've not investigated
this much, but implemented this to preserve as much of the TARGET_ABI
functionality that I could. Since its presence doesn't affect the
working cases, I've kept it in for now.

Added mips64e[lb] to make universe, so more kernels build.

And I think this (finally) closes the curtain on the tbemd tree.

When TARGET_ARCH_${kernel} gets set, it is using the host's config,
not the one we build as part of make world. This means that make
universe will fail if building on a too-old current or any stable
system prior to a few days ago in weird ways (parse errors from
shell). This copes with these old systems in two ways:

(1) Works around the WARNING: issue by filtering all warnings that
sneak onto stdout.

(2) if TARGET_ARCH_${kernel} winds up being empty, then we error out
immediately with a semi-useful error message. This usually comes from
config not groking -m.

Ideally, we'd use a buildworld's config here, but that's tricky, so
I'll leave that detail to others to fix (it has to be done post make
world for the arch rather than at the top level makefile). This
should make 'make universe' usable from recent 8-stable systems
(recent == last few months or so) for building -current. They have
-m, but spewed warnings out stdout. Older systems will now at least
get a firm error early rather than a confusing error late.

Redirect stderr from config to /dev/null. config -m is printing lots
of annoying warnings when dealing with arm. The arm config files need
to be fixed, but this restricts the output to a more useful place.

Reconnect arm to the universe build, and connect big-endian MIPS and ARM
and powerpc64 to universe for the first time. In general, provide
(slightly hacky) knowledge of multi-architecture TARGETs to universe as
well as the ability to distinguish the correct toolchain for a given
kernel using config -m.

Pass JFLAG as JFLAG from tinderbox to universe. This gives the same
semantics for JFLAG with tinderbox as for universe. Previously doing
'make JFLAG=-j4 tinderbox' was equivalent to 'make -j4 universe'
(i.e. 4 worlds in parallel) rather than 'make JFLAG=-j4 universe'
(i.e. worlds in sequence, each built with -j4).

MFC after: 1 month

Add mips back to universe

Complete the integration of tbemd branch into head.

TARGET_BIG_ENDIAN is now completely dead, except where it was
originally supposed to be used (internally in the toolchain building).

TARGET_ARCH has changed in three cases:
(1) Little endian mips has changed to mipsel.
(2) Big endian mips has changed to mipseb.
(3) Big endian arm has changed to armeb.

Some additional changes are needed to make 'make universe' work on arm
and mips after this change, so those are commented out for now.

UPDATING information will be forthcoming. Any remaining rough edges
will be hammered out in -current.

Connect FDT infrastructure to the build system.

Reviewed by: imp
Sponsored by: The FreeBSD Foundation

For those of us mere mortals who do not aspire to the lofty heights
of kernel hackery, add MAKE_JUST_WORLDS so that we can take part in
the 'make universe' goodnes without using unecessary time and resources.

Give make kernel-toolchain a one-liner here. Although documented in
build(7), many people look here for a quick reminder...

Back out the change to Makefile made in r202628.

It was just a local modification to build the i386 and amd64 more
quickly.

Recommit r193732:

Remove __gnu89_inline.

Now that we use C99 almost everywhere, just use C99-style in the pmap
code. Since the pmap code is the only consumer of __gnu89_inline, remove
it from cdefs.h as well. Because the flag was only introduced 17 months
ago, I don't expect any problems.

Reviewed by: alc

It was backed out, because it prevented us from building kernels using a
7.x compiler. Now that most people use 8.x, there is nothing that holds
us back. Even if people run 7.x, they should be able to build a kernel
if they run `make kernel-toolchain' or `make buildworld' first.

Rather than using an extra variable, only call uname if really needed and
then directly assign the result.

Submitted by: jmallett
MFC after: 24 days
X-MFC with: r201815

Use uname -m [1] and rename BUILD_ARCH to XMACHINE[2].

Submitted by: nyan[1], imp[2]
MFC after: 27 days
X-MFC with: r201815

To avoid hardcoding further kernel configuration names for
make universe, split the logic into two parts:
- 1st to build worlds and generate kernel configs like LINT.
- 2nd to build kernels for a given TARGET architecture correctly
finding all newly generated configs, not knowing anything about
LINT anymore. (*)

(*) If you know better/cleaner/... ways to do this, let me know.

Discussed on/with: arch, jhb, rwatson
MFC after: 1 month

Add a new world named 'mips' to our universe.

Discussed with: imp

Implement the xdev target. When you define XDEV=arch XDEV_ARCH=arch,
you can build the cross development tools and install them as
$XDEV-freebsd-xxx for each tool. This allows one to use autoconf to
find the tools for cross building scenarios.

Permit digits at the beginning and end of kernel config file names for
`make universe'. This catches a few more arm and, once enabled, mips
configs and permits having local configs named like NOINET6.

Reviewed by: phk
MFC after: 4 weeks

Provide a 'tinderbox' target that compiles enough of FreeBSD that
a developer can rest reasonably assured that the tinderbox will not
be broken. This target leverages most of 'universe' but will exit
non-zero and output a summary at the end.

"make tinderbox"

Change the universe target to warn the user for every world or kernel that
fails. The error message includes a reference to the relevant log file.

Approved by: phk
MFC after: 1 week

Back out rev. 1.352 (SVN rev 179842) as phk pointed out that
SRCCONF was omitted here to be able to build non standard trees
with universe as well.

For make universe, in addition to make.conf, also ignore a
src.conf with possibe non-default options.

Reviewed by: ru
MFC after: 10 days

Backout the change to remove sun4v.

Note that sun4v is currently broken on T2000 hardware so it's pretty
much a lame duck at this point.

Remove sun4v from the list of arches in 'make universe'. There has been
no active development on it for over a year now and it isn't
reliable under a simple buildworld. Developers can't be expected to
test code targeted for it.

Add a knob to allow just the kernels to be built during a 'make universe'.
This is useful when wanting to retry a broken build using NOCLEAN and the
buildworlds are known to have completed.

Mention -U and -ai arguments to mergemaster in a comment for the
general mergemaster line.

MFC after: 3 days
Suggested by: Ben Laurie

Double up the necessary size of /usr/obj to 800MB

PR: misc/99735
Reported by: Philip M. Gollucci <pgollucci@p6m7g8.com>,
Maxim Konovalov <maxim@macomnet.ru>

Add code that should catch the most common case when date/time is
set incorrectly.

Add a note pointing to build(7) for more info.

MFC after: 1 week
Reviewed by: ru

We don't need to invent BSDMAKE. ${MAKE} is sufficient.

Submitted by: jmallet@

Don't assume that make(1) is a bsd-like make program. It might not be. This
only matters in the early stages of bootstrapping, of course, but gnu make can't
handle bsd make Makefiles at all if they use any of the 'dot' directives, which
src/Makefile has in abudnance.

Do not bail out for cleanworld target just because
chflags is failed.

Expose all of {check,delete}-old{,-dirs,-files,-libs}.

MFC after: 3 weeks

Introduce a variable TARGETS that can be set on the make command line
and that controls which platforms are being built as part of a "make
universe". By default TARGETS is set to the 8 platforms currently
being built. This variable is useful for running or re-running a
"make universe" with only a selected set of platforms. This makes the
universe target slightly more useful in cases the universe is limited
to a developer's scope or objectives. For example, when a universe
failed for a particular platform and fixes need to be tested for that
particular platform then a developer can restart the universe for
only that platform, even if the initial universe is still building
other platforms.

Add arm to universe.

Remove the NO_CTF option. It would have been required to support
DTrace.

When building an upgraded make, don't worry about processing it for
use with DTrace because the normal buildworld will do that when the
tools are built.

Add sun4v to the 'universe'.

In "make universe", utilize simplified cross-build synopsis,
"make TARGET=<machine> build*", now that it actually works.

Prompted by: imp

Per weak consensus on this topic, remove suggestion to set
HISTORICAL_MAKE_WORLD from the text that's output. This was committed
against the previous consensus. Leave the documentation in this file
as a compromose. The HISTORICAL_MAKE_WORLD knob is intentionally
obfuscated and we only trust people smart enough to read the Makefile
to use it. All others have no business using it due to its danger,
unless DESTDIR is set.

Dissentors: grog, obrien, trhodes

Note a short-cut in the build procedure.

Simply running ``make world'' will bomb unless you dig up the
undocumented HISTORICAL_MAKE_WORLD variable and set it. Note it
here so the blow up will not really be a surprise to people who
read.

Link the buildingworld chapter of our handbook in the README
while I'm here.

Create a new target 'buildenvvars'. This target reports the build
environment for cross building (the same one you'd get interactively
in make buildenv). This cannot be a simple

make -f Makefile.inc1 -V WMAKEENV

because in PATH is not set correctly unless one takes a trip through
the Makefile/Makefile.inc1 indirection, the logic of which is too
large to reproduce outside of Makefiles.

Connect the kernel subsystem API documentation generation framework to
the build infrastructure: "make doxygen"

Changes to the man pages will be made when some defaults are revisited.

Requested by: gnn

Disconnect Alpha from 'make universe'.

Add the new "showconfig" target that displays build configuration.

Suggested by: phk

Make ${SUBDIR_TARGETS} work at the uppermost level as well.

Do not discard the current value of __MAKE_CONF when testing
whether MAKEOBJDIRPREFIX is set at a wrong place.

Reviewed by: ru
MFC after: 3 days

Fix a bug in previous revision: skip LINT if it exists, not NOTES
(the source for LINT), as I initially intended.

universe: don't attempt to compile DEFAULTS and don't compile LINT twice.

Add delete-old and delete-old-libs targets:
- removes obsolete files/dirs or libraries.
- works in interactive (default) and batch mode
- respects DISTDIR
- documented in UPDATING and build(7)

The head of the file ObsoleteFiles.inc contains instructions how to add
obsolete files/dirs/libs to the list. Obviously one should add obsolete
files to this list, when he removes a file/dir/lib from the basesystem.

Additionally add check-old target:
- allows re@ to check if a file on the obsolete list resurfaces

Design goals:
- allows full control by the user (default interactive mode)
- possibility of scripted removal of obsolete files (batch mode)
- opt-in removal of files (explicit list of files)
- seperate removal of libs (2 delete targets)

Important design decissions:
- structured list of files to remove instead of a plain text file:
* allows to remove additional files if a NO_foo knob is specified
without the need to change the targets (no NO_foo knob is respected
yet)
- not using mtree like NetBSD does:
* mtree doesn't has an interactive mode

Discussed on: arch (long ago), current (this year)
Additional input from: re (hrs)
Approved by: mentor (joerg)

- Embed the "buildkernels" target into the "universe" target so that
the "make -n universe" output looks more builder (human) friendly.

- Wrap the "universe" target into a ".if make(universe)"; it's only
intended to be called directly so it should be safe to do it.

Make the `distrib-dirs' target official and working properly
when cross-building (installing). (This is useful for NFS
installing world/kernel to another architecture machine.)

Provide 32-bit runtime support on amd64 as a separate distribution, lib32.

Prodded by: obrien
Nodded by: peter
Approved by: re

Repo-copy tools/regression/usr.bin/make to tools/build/make_check
and adjust the path in the Makefile for the upgrade_checks target.
These checks are really feature upgrade checks that should be fast
and just find out whether we need to build a new make before
proceeding with other targets like buildworld. This makes the
place free for a real regression test suite in the old place.

Added the convenience "distribution" target which calls the
target of the same name from src/etc/Makefile with a proper
environment, suitable to be used during upgrades and cross-
builds.

Added the "buildenv" target, for developers only!

CAVEAT: if you run tcsh(1) from within this target, this will
reset your PATH making this target mostly useless. Careful!

Submitted by: jmg, ru

NOSHARED -> NO_SHARED

Start the dreaded NOFOO -> NO_FOO conversion.

OK'ed by: core

Previous revision didn't fix all cases. For example,
testing for variables that are always defined (e.g.,
"make -V CC") would still print a false warning. Fix
this by only passing a submake the MAKEOBJDIRPREFIX=*
argument if it was present. As a result, we loose
the check for -DMAKEOBJDIRPREFIX, or an esoteric
"MAKEOBJDIRPREFIX = foo" (with embedded spaces), but
these are unorthodox enough to not care about them.

The make(1) bug mentioned in the previous revision
was just fixed in make/main.c,v 1.109.

Unbreak the MAKEOBJDIRPREFIX variable check by avoiding bugs in make(1).
When make flags are passed to make in the environment, the string is
chopped up in an (argc,argv) vector. This happens in brk_string() and
the chopped up string is stored in static buffer. When this includes
something like "-V BINMAKE", then a pointer into the static buffer is
put on the variables list for evaluation later. However, brk_string()
is used for more than just chopping up the MAKEFLAGS env. variable, so
it's very likely that the static buffer is clobbered. In fact, this is
exactly what happens.
The result is that _MAKEOBJDIRPREFIX gets assigned whatever garbage
the child make happens to emit, causing the test to fail. Like this:

pluto2% cd /usr/src
pluto2% make -V BINMAKE
"/q/6.x/src/Makefile", line 94: MAKEOBJDIRPREFIX can only be set in environment, not as a global (in /etc/make.conf) or command-line variable.
pluto2% make -dv -V BINMAKE | & grep _MAKEOBJDIRPREFIX
Global:_MAKEOBJDIRPREFIX = }

The fix is to not use MAKEFLAGS for this, but simple pass the flags
as arguments. Ideally make(1) should be fixed but that's beyond the
scope of my attention span.

This fixes release.

Fix builds with a read-only directory and a make upgrade. This is done
by forcing the creation of an object directory for the make regression
tests. Let make handle the tracking of the dependency and installation
of test_shell script.

Submitted by: ru

Convert tools/lib32/build.sh into world connectable hooks. This still
rates pretty high on the "hack!" scale, but it works for me. Adding
-DWANT_LIB32 to the world build command line, or 'WANT_LIB32=yes' to
/etc/make.conf will include the 32 bit libraries with the build.

I have not made this default behavior. Cross compiling this stuff is an
adventure I have not investigated.

This is still a WIP. We needed this at work so that we could install from
a readonly obj tree - lib32/build.sh wasn't up to that.

Fixed MAKEOBJDIRPREFIX variable type check in case the env(1)
binary is not in PATH, as is the case with installworld, etc.

My take at improving the universe: allow the worlds to be
built in parallel. Examples:

make universe
Build worlds sequentially, each world sequentially.

make universe JFLAG=-j4
Build worlds sequentially, each world in parallel.

make -j4 universe
make -j4 universe JFLAG=-j2
Build four worlds in parallel, each world will be
built in parallel too. World parallelization is
set to four in the first synopsis, and to two in
the second.

make -j4 universe JFLAG=-B
Build worlds in parallel, each world sequentially.

("world" == buildworld followed by buildkernels.)

Prayers: obrien, phk

Moved the MAKEOBJDIRPREFIX check from Makefile.inc1 to Makefile,
to suppress warnings with installworld and distributeworld when
env(1) cannot be found in the PATH.

Make make recurse into sub-directories and sub-makes when given
two -n flags. If only one -n flag is given the old behaviour
is retained (POLA). In order to make this working for installworld
change the IMAKEENV in this case so that the tools are found
(we have no temporary installation environment in this case).

Submitted by: ru (IMAKEENV part)

The doc team reworked the section of the handbook describing how to
rebuild things, further discouraging 'make world'. The title of
that section is now changed...

Rename a preposterously long name with one that might look a bit
more professional. While here, write a few lines of explanatory
text to explain what its for.

Discussed with: rwatson
With hat: core

Add an "out" to the now required BS of defining DESTDIR.

Properly terminate a quoted string in the previous commit. Oddly
enough this only seems to cause problems with -jX so went unnoticed
until now.

As per discussions on current@, protect unsuspecting users from
trying to upgrade their system with make world instead of following
the preferred and suggested sequence of commands. The fact remains
that make world does not upgrade the kernel.
Allow make world when DESTDIR has been specified, including when
DESTDIR specifies the root file system. Otherwise, print a useful
warning and fail.

Reviewed, tested and scrutinized by: gad@

Add a JFLAG which can be used to speed up universe:
make universe JFLAG=-j12

Add a kernel-toolchain target which only builds the bits required to build
a kernel. This is essentially the same as the toolchain target, except
that it does not build headers and libraries.

Submitted by: ru

Added the `toolchain' top-level target, which builds enough of
buildworld, up to and including libraries, except for actually
building the world.

Requested by: many

Protect shell expansions.

Add amd64 to universe.

Approved by: peter

Change the 'realclean' target name to be 'cleanworld', to match other
targets like buildworld, installworld.

Suggested by: obrien

Improvements on the 'make realclean' target. Some are style improvements,
but the biggest issue is that there are situatons when
${.OBJDIR} == ${.SRCDIR}, and in those situations the previous version
would happily remove all your /usr/src while it was cleaning out the objects.
Not that *you* would be happy about it... Thanks to bde for immediately
noticing this serious possibility.

More improvements will be made to this target, but I wanted to commit this
safer version right now, before anyone lost their /usr/src due to it.

Reviewed by: bde
MFC after: 10 days

Add a 'realclean' target, which simply and quickly removes everything
that was built in previous 'make buildworld' and 'make buildkernel'. The
target knows enough to run a 'chflags -R 0' and a second 'rm' if the first
'rm' ran into any errors while removing files.

Suggested by: email with Richard Coleman Re: upcoming 64b-time_t changes.
Reviewed by: imp, marcel, and others on -hackers
MFC after: 1 week

Put on some factor 30+ bikeshed repellent and export the internal
Makefile.inc1 underscore targets with a big warning that they are
intentionally undocumented, subject to change without notice and
might poison your dog etc. If you don't know what they are, then they
are not meant for you to use.

I've added these by hand to so many many trees that I've lost count. I
find them rather useful.

Don't allow the 'make' target (which rebuilds make for bootstrapping)
to become confused by a file called 'make' in /usr/src

Submitted by: ru

Retired the "most" and "installmost" targets -- they just
do not have a chance to work nowadays as we have a lot of
internal libraries in lib/.

Discussed with: marcel, wollman

Cosmetics: folded BITGTS into TGTS.

Limit testing for an up-to-date version of make(1) to the world and
buildworld targets by default, but allow it to be done for all user
targets by introducing a boolean option, named ALWAYS_CHECK_MAKE.

This change is by no means perfect and I don't even want to claim
this to be a solution. It does however address the fact that not
everybody likes to see make(1) rebuilt simply because the regression
test failed for some reason or other, including pilot error. It
therefore serves the purpose of keeping the crowd happy until we
have something better or simply reached a compromise.

The reasons for changing the default behaviour are:
o It avoids a negative, possibly non-intuitive option,
o It's according to POLA and fond of feet,
o Only buildworld is documented to do its best to be
successful at reasonably cost.

Reviewed by: gad, imp, obrien, peter

- Don't print "elf" before printing "make world" (aout support has been
removed).

- Avoid using printf(1) where a simple shell's built-in echo is enough.

The `mk' target is long dead.

Use an up-to-date make binary in the upgrade_checks target if
one is already available. This avoids sometimes unnecessary
step of attempting to rebuild the make binary again which may
fail at all if, for example, one has removed his /usr/include
before doing an installworld (to keep /usr/include tidied up).

Pointed out by: kris, marcel

Don't hardcore PATH into BINMAKE; the latter should only be
comprised of the path to a make(1) binary (possibly the one
built by the "make" target in this Makefile), and a path to
a fresh share/mk. The idea is to allow "make release" pick
up the right "make" binary, if one exists.

This fixes release.5 I broke with the last commit here; the
second PATH here was overriding the right one from WMAKEENV
while rebuilding the "build-tools" for crunched binaries.

Fixed style bugs related to parentheses in Makefile.inc1.

Submitted by: bde

Fixed nearby bug: propagate the root Makefile's idea of
the appropriate "make" binary down to release/Makefile.

Do the upgrade checks quietly.

Allow installkernel.debug and reinstallkernel.debug.

Kill upgrade target docs. It too is stale.

Submitted by: ru

Kill aout to elf upgrade path. From an old comment in the makefile:

# If -DWANT_AOUT is specified, a `make world' with OBJFORMAT=elf will
# update the legacy support for aout. This includes all libraries, ld.so
# and boot objects. This part of build should be regarded as
# deprecated and you should _not_ expect to be able to do this past the
# release of 4.0. You have exactly one major release to move entirely
# to elf.

Now that we're past 5.0, these aren't needed at all anymore and it is
well past its freshness date.

Don't make buildworld and buildkernel in parallel.
And this is probably what you expect when you say
"make -jX buildworld buildkernel".

Don't spam sys/${MACHINE}/conf/ with _.${MACHINE}.makeLINT.

Submitted by: johan

Fixed universe.

Folded pc98 into the common case.
Retired ${JFLAG} (``make -jX universe'' should work).

Fix make universe's kernel builds on pc98: We need to pass both ARCH
and TARGET_ARCH.

Part 3/3 of unbreaking cross releases:

When we call "distributeworld" as part of "make release", we set
MACHINE and MACHINE_ARCH to point to TARGET and TARGET_ARCH; this
confused src/Makefile's idea of what ${MAKEPATH}, and consequently
${MAKE}, is (well, it still confuses ${MAKEPATH}, but see below).

To overcome this problem, we now take the following approach:

- We preserve the make(1)'s idea of its argv[0], ${MAKE}.
- We check to see if ${MAKE} passes the regression tests.
- If it does, we use it. Otherwise, we build and use an
up-to-date make(1).

This fix is still not quite right, in a situation where a single
/usr/obj is shared between different architecture machines, but
it is less critical and I hope to fix that soon.

Minor typo fixes and whitespace cleanups.

Build kernels for pc98 just like the other platforms. Also, remove
one space to make the dates line up for easy mental math.

# Too bad only i386 and pc98 are the only parts of the universe
# currently buildable on my box :-(

Minor, cosmetic change to buildworld output to make it reflect
rev. 1.270 more accurately.

Reviewed by: ru
Approved by: re (jhb)

If /usr/bin/make fails to pass the regression tests, it might not
be suitable for building. The old action here was to upgrade the
make in /usr/bin. This is both bogus and does not work under non
root. So, rather than overwriting the existing /usr/bin/make, we
now install it into a safe location, and use it from there.

Reviewed by: imp, marcel
Approved by: re (rwatson)

Moved make(1) regression tests from src/Makefile to where they
belong (src/tools/regression/usr.bin/make), and use the latter
to test if make(1) is adequate for building the world.

Approved by: re

PC98 is not a true architecture, but a subarchitecture of i386, so we need
to use a slightly differnt command to PC98::buildworld.

Add pc98 to universe target.
Remember about /usr/obj and look for conf/NOTES in the right place.

Add "make universe" target which:

for each arch i386 sparc64 alpha ia64
make buildworld TARGET_ARCH=$arch
if NOTES found
make LINT
foreach kern ${arch}/conf/[A-Z0-9]*
make buildkernel TARGET_ARCH=$arch KERNCONF=$kern

Errors from the individual steps are ignored and the stdout+stderr
are saved in distinct files in root of the source tree with names
like: _.${arch}.${target}

On my dual 1.6G Athlon it takes roughly four hours to run this target.

Initiate deorbit burn for the i386-only a.out related support. Moves are
under way to move the remnants of the a.out toolchain to ports. As the
comment in src/Makefile said, this stuff is deprecated and one should not
expect this to remain beyond 4.0-REL. It has already lasted WAY beyond
that.

Notable exceptions:
gcc - I have not touched the a.out generation stuff there.
ldd/ldconfig - still have some code to interface with a.out rtld.
old as/ld/etc - I have not removed these yet, pending their move to ports.
some includes - necessary for ldd/ldconfig for now.

Tested on: i386 (extensively), alpha

Deal with bootstrapping from an old -current (almost exactly a year old)
which fails the make tests (doesn't understand ${notdef:U}) and therefore
fails on __FBSDID in usr.bin/make/*. -DBOOTSTRAPPING is no help here since
this is before we are using the new share/mk/* files, and it would conflict
with the builtin -DBOOTSTRAPPING support later.. so use a different flag.

Indentation.

Upgrade make(1) if not passing the ``.if defined(notdef) && ${notdef:U}''
test (revealed by bsd.lib.mk,v 1.134).

correct syntax in last commit

Document TARGET_ARCH

Reimplement FILES support using bsd.files.mk with the
same set of features as in recently added bsd.incs.mk
(FILESGROUPS, accessibility from both bsd.prog.mk and
bsd.lib.mk, de-pessimized typical installation path,
etc.) New standard targets: buildfiles, installfiles,
and files (buildfiles + installfiles).

Line up the make world completed output.

Forgot to commit this for the `includes' -> `buildincludes',
`incsinstall' -> `installincludes' changes.

Added new bsd.incs.mk which handles installing of header files
via INCS. Implemented INCSLINKS (equivalent to SYMLINKS) to
handle symlinking include files. Allow for multiple groups of
include files to be installed, with the powerful INCSGROUPS knob.
Documentation to follow.

Added standard `includes' and `incsinstall' targets, use them
in Makefile.inc1. Headers from the following makefiles were
not installed before (during `includes' in Makefile.inc1):

kerberos5/lib/libtelnet/Makefile
lib/libbz2/Makefile
lib/libdevinfo/Makefile
lib/libform/Makefile
lib/libisc/Makefile
lib/libmenu/Makefile
lib/libmilter/Makefile
lib/libpanel/Makefile

Replaced all `beforeinstall' targets for installing includes
with the INCS stuff.

Renamed INCDIR to INCSDIR, for consistency with FILES and SCRIPTS,
and for compatibility with NetBSD. Similarly for INCOWN, INCGRP,
and INCMODE.

Consistently use INCLUDEDIR instead of /usr/include.

gnu/lib/libstdc++/Makefile and gnu/lib/libsupc++/Makefile changes
were only lightly tested due to the missing contrib/libstdc++-v3.
I fully tested the pre-WIP_GCC31 version of this patch with the
contrib/libstdc++.295 stuff.

These changes have been tested on i386 with the -DNO_WERROR "make
world" and "make release".

Bring doc up-to-date with current buildworld/buildkernel practices.

Reviewed by: jhb

Added internal, non-recursive (SUBDIR) versions of the `all-man'
and `maninstall' targets. This fixes the issue where each subdir
was descended into twice during "make all", and also resurrects
the standardization of `maninstall'.

Urged by: bde

Milestone #2 in cross-arch make releases. They now work!

You need to set TARGET_ARCH and possibly TARGET, the same
way you normally do it for a cross build(7).

Renamed `distribworld' to a more natutal `distributeworld'.

Put pwd_mkdb(8) under ${INSTALLTMP}; for `distributeworld'.

Parallel "make release" fixes.

Milestone #1 in cross-arch make releases.

Do not install games and profiled libraries to the ${CHROOTDIR}
with the initial installworld.

Eliminate the need in the second installworld. For that, make sure
_everything_ is built in the "world" environment, using the right
tool chain.

Added SUBDIR_OVERRIDE helper stuff to Makefile.inc1. Split the
buildworld process into stages, and skip some stages when
SUBDIR_OVERRIDE is set (used to build crypto, krb4, and krb5
dists).

Added NO_MAKEDB_RUN knob to Makefile.inc1 to avoid running
makewhatis(1) at the end of installworld (used when making crypto,
krb4, and krb5 dists).

In release/scripts/doFS.sh, ensure that the correct boot blocks are
used.

Moved the creation of the "crypto" dist from release.5 to
release.2.

In release.3 and doMFSKERN, build kernels in the "world"
environment. KERNELS now means "additional" kernels, GENERIC is
always built.

Ensure we build crunched binaries in the "world" environment.
Obfuscate release/Makefile some more (WMAKEENV) to achieve this.

Inline createBOOTMFS target.

Use already built GENERIC kernel modules to augment mfsfd's
/stand/modules. GC doMODULES as such.

Assorted fixes:

Get rid of the "afterdistribute" target by moving the single use
of it from sys/Makefile to etc/Makefile's "distribute".

Makefile.inc1: apparently "etc" no longer needs to be last for
"distribute" to succeed.

gnu/usr.bin/perl/library/Makefile.inc: do not override the
"install" and "distribute" targets, do it the "canonical" way.

release/scripts/{man,cat}pages-make.sh: make sure Perl manpages and
catpages appear in the right dists. Note that because Perl does
not respect the MANBUILDCAT (and NOMAN), this results in a loss of
/usr/share/perl/man/cat* empty directories. This will be fixed
soon.

Turn MAKE_KERBEROS4 into a plain boolean variable (if it is set it
means "make KerberosIV"), as documented in the make.conf(5)
manpage. Most of the userland makefiles did not test it for "YES"
anyway.

XXX Should specialized kerberized libpam versions be included into
the krb4 and krb5 dists? (libpam.a would be incorrect anyway if
both krb4 and krb5 dists were choosen.)

Make sure "games" dist is made before "catpages", otherwise games
catpages settle in the wrong dist.

Fast build machine provided by: Igor Kucherenko <kivvy@sunbay.com>

Merge bsd.obj.mk's version of the _SUBDIR target with bsd.subdir.mk.

Ensure all standard targets honor SUBDIR. Now `make obj' descends into
SUBDIRs even if NOOBJ is set (some descendants may still need an object
directory, but we do not have such precedents). Now `make install' in
non-bsd.subdir.mk makefiles runs `afterinstall' target _after_ `install'
in SUBDIRs, like we do in bsd.subdir.mk. Nothing depended on the wrong
order anyway.

Fixed `distribute' targets (except for the bsd.subdir.mk version) so that
they do not depend on _SUBDIR; `distribute' calls `install' which already
depends on _SUBDIR.

De-standardize `maninstall', otherwise manpages would be installed twice.
(To be revised later.)

Despite the fact that it is documented in the handbook, the release notes,
and UPDATING and has been posted to both freebsd-current and
freebsd-stable, users are still not adding the required smmsp user and
group before doing an installworld. Therefore, don't let users do an
installworld unless they have followed directions.

Add a new installcheck Makefile target which installworld runs before
actually starting the installation. This target can be used by other parts
of userland as well. The first addition to the target is to check for the
smmsp user and group if NO_SENDMAIL isn't defined.

Others may add checks to this target as they see fit.

MFC after: 1 week

Makefile.inc0 is no longer used.
Update comments to reflect that.

Approved by: dd, silence in -audit.

FreeBSD 4.1 bootstrapping aid (``:C'' make(1) modifier used
by gnu/usr.bin/binutils/as/Makefile.inc0 is not there).

XXX: make(1) should return with a non-zero exit status here.

Make it possible to build manpages for the entire source tree.

End words with a `.'.
Fixed spacing messed up in rev 1.244.

setlocale(3) has been fixed to match POSIX standard:
LC_ALL takes precedence over other LC_* envariables.

Cosmetic fix: Document targets at the top in a uniform way.

Add back displaying the `world' start time when displaying the stop time.
However, don't prefix it with '>>>', so those greping for times will
not see it.

Also list kernel target so one can get to it.

Document the "kernel" target.

Fix release, broken by the perl cross-build fixes.

The distribute target is basicly the same as an install. For
perl, this means that miniperl is needed. Since miniperl is
only present in the object directory, we need to make sure
the path is set correctly. To do this, we have make release
use a new distribworld target that sets the path before doing
a make distribute.

While we're still arguing over debugging output, at least be in a
self-consistent state.

Requested by: bde

MFS: eliminate duplicate start line

Pass the reinstallkernel target through from Makefile to Makefile.inc1.

Mention the buildkernel/installkernel targets in the documentation.

We have a new world order in libraries.

Goodbye libdes; Welcome libcrypto.

Add the buildkernel and installkernel targets.
While here fix some minor style bugs (whitespacing) and move the
make target from Makefile.upgrade to this file.
Simplify the make target to make it readable.

Descent into sub-makes with a fixed PATH. The PATH does not contain
/usr/games on purpose. All tools in /usr/games are built as part of
the tool stages to support building world on machines without games.

This fixes any build problems that have been caused by non-standard
paths and also removes the primary objection for not using absolute
paths.

Oops! bypassing Makefile.inc0 without adding a `world' target is not
a good idea. Adding `world' to Makefile is save, because all hairy
bits are in Makefile.inc1 anyway.

Restore the upgrade path from -stable to -current and prepare for
non-root cross-building.

o Makefile.inc0 is not used anymore.
o The legacy aout build has been removed.
o Selectively build tools *before* building includes/libraries.
o Avoid using mtree to populate the obj tree.

Remove a comment about setting objformat to elf by default. We've
done that already.

Change the -DWANT_AOUT comment to refer to 4.0 so that it sounds less
silly.

$Id$ -> $FreeBSD$

Comment change: "Be warned, this will update your installed system, except
for the configuration files in the /etc directory." .... and the kernel.

Change !defined(NOAOUT) knob to defined(WANT_AOUT).

Note that the upgrade target exists, and what it does.

Requested by: Wayne M Barnes <stabilizer@klentaq1.emergingtech.org>

Use ${MAKE} not 'make -f'

Submitted by: tfreak@fluffybunny.jaded.net

Frob the upgrade target to be a bit more inclusive. This appears to
be a good bit more successful at doing automated upgrades than the
previous aout-to-elf target, anyway. I'll MF3 in a couple of days
depending on how this does.

Dip my toes into the fire and zap the leftover lkm hooks.. It seems they
try and recurse if the lkm dir exists for some reason but there isn't any
Makefile there. (eg: stray files prevented cvs update -P from removing the
empty dirs)

Restored all lost user targets that are supported by bsd.subdir.mk:
checkdpadd, lint, maninstall, objlink, regress and tags.

Removed bogus user target cleanobj. It is the non-recursive base of
the cleandir target, so it is not useful (or usable) here.

Add back a few useful targets lost in the reshuffle.
Reviewed by: jb

The legacy stuff needs gobs more space in the obj tree. 165->260 Mb.

Restore default implict behavior of running the all target we had before.

Revive hierarchy again.
Please commit only patches, not whole files!

Add the missing rerelease target back.

Reported by: Justin Gibbs.

Add the move-aout-libs upgrade target so that people who have already
gone elf can put their libraries through the mincer. Anyone who hasn't
deleted aout libraries from /usr/lib (but has done a make world putting
the new aout libs in /usr/lib/aout) will be asked for confirmation
to delete them one by one.

Revive hierarchy target

YAMT (yet another missing target). It's a shame that these have to
be visible to the user. Maybe `make release' should call the backend
directly.

Add the distribute target that make release wants.

Add the clean cleandepend and cleanobj targets back.

E-day build system changes.

- Moved most of the guts of Makefile to Makefile.inc1 to become the
backend for the build system.
- The new Makefile doesn't suffer from problems including the wrong
sys.mk because it doesn't use anything in there or bsd.own.mk. So,
from now on, the proper build command is just `make world' (or
buildworld).
- The intermediate makefiles called Makefile.inc0 and Makefile.upgrade
fiddle with the OBJFORMAT and MAKEOBJDIRPREFIX variables so that
both aout and elf object trees can coexist. Makefile.upgrade contains
the aout->elf transition build.
- A cross build environment is now very close to reality. Specifying
TOOLDIR, setting OBJFORMAT and MAKEOBJDIRPREFIX allow that.

See the comments in Makefile for more info.

Build objformat on all architectures.

Fully enable the optimization of not building dependencies unless NOCLEAN
is set. It was disabled in the -j case. See rev.1.173.

Descend into etc always; src/etc/sendmail/Makefile has make.conf hooks
for building and installing a local sendmail.cf..

I'm a little nervous about the implications of having an obj dir built
under etc (to get to the obj dir for sendmail), but the make rules appear
to DTRT.

Fixed building -current under 2.2.6 using `make world'. Moved some
recently added definitions from sys.mk to bsd.own.mk. Include the
src-relative bsd.own.mk in src/Makefile to pick up all new definitions.
Don't check that MACHINE_ARCH is defined in src/Makefile, since it is
(and should have been) guaranteed to be defined.

Malformed conditional if MACHINE_ARCH is not defined. From rev 1.195.

Oops, don't build tools for building games, etc. when we're not building
games, etc.

Define _BUILD_TOOLS in sub-makes for building tools. This will be used
to avoid using uninstalled tools in colldef and mklocale.

Build internal tools in build-tools so that they have some chance of
working when the target system is not binary compatible. Use various
hacks to work around minor problems in the source and binary tree
layouts:
- caesar and strfile are built normally (the source layout is good),
then installed by copying them to ${WORLDTMP}/usr/bin (they are
installed in ${WORLDTMP}/usr/games, but I don't want to put that
in $PATH).
- colldef and mklocale are built and installed normally. Messy and
incomplete relative path searches for them and caesar and strfile
can now go away.
- internal tools that aren't installed are now built and left lying
around for the `make all' pass to use. If the target system is
not binary compatible, it is critical that these tools don't get
rebuilt. Cleaning of the obj tree before building the internal
tools should ensure this.
- most internal tools are built using internal build-tools targets,
but tn3270 is simpler for a change - it has all the tools in a
separate tree, so they can be built using `make all'.

Added yacc to the bootstrap tools. It is needed very early to
build at least compile_et and lex, and although almost any version
of yacc could work, the version in -stable doesn't actually work
with -current makefiles because it doesn't support -o.

Submitted by: Ian Holland <ianh@tortuga.com.au>

join(1) is used by lorder, add it to build-tools. (otherwise, things get
ugly when an a.out /usr/bin/join sees the elf LD_LIBRARY_PATH from a
'make world' and it's a.out ld.so tries to load the elf libc.so.)

Quick fixes for the mtree bootstrap:
- don't announce `mtree' as `mtools'.
- don't install to ${DESTDIR}/usr/sbin (which often doesn't exist if
DESTDIR is set and may be read-only if DESTDIR is not set).
- install to (${WORLDTMP}/usr/sbin so that the new mtree is actually
in $PATH if DESTDIR is set.
- don't use the host make or the host sys.mk. This is probably
unimportant.
- use a temporary obj dir like the one for `make'. This was mainly
necessary because I forgot to remove the MAKEOBJDIRPREFIX unsetting
which was just a bug for mtree. A non-quick fix would handle mtree
more like a bootstrap tool (the only additional complications are to
create ${BINDIR} and avoid excessive cleaning). Except a non-quick
fix would change much more.

Compile mtree before it is used, this is a hack, but make world has
failed since new functionality was introduced in mtree.

-DALLLANG is now obsolete in "src/Makefile"; move to "src/release/Makefile".

PR: 6685

Don't run the target's ldconfig or build it as a tool. If target == host
(in particular, if DESTDIR is empty or "/"), then the host's ldconfig will
be the target's ldconfig by the time it is run.

Fixed disordering of env.

Don't know too much about libcrypt. Use exactly the same definition of
_libcrypt as lib/Makefile.

Don't build strip twice.

Fixed bogotification of the lex bootstrap in rev.1.186 and the previous
commit - don't wander off to bootstrap mtree and include in the middle
of bootstrapping lex, and don't forget what we were doing and build
some lex obj dirs twice.

Add mtree to bootstrap targets now that peter has added a new
keyword to it which will otherwise call the next target to fall over
on transitioning systems.

Build aout bootstrap tools only on i386.

`make world' now works on alpha!

Typos.

Don't build perl in build-tools: if NOPERL is defined.

Fix brain-fart. "usr.bin/reinstall" should be "sbin/ldconfig".
No comment. *-<:-)

Submitted by: "Pierre Y. Dampure" <pierre.dampure@k2c.co.uk>

ldconfig is used at the end of installworld.

Add a test to see if the secure directory exists, before deciding which
crypt library should be used when building bootstrap-libraries. This
make it work on machines that don't have the secure directory.
Thanks to Paul Allenby <pallenby@mikom.csir.co.za> for bringing it to
my attention.

Whoops! Committed the wrong version of that file - here's the one which
works.

Move sh after awk in build tools - sh requires awk to build. Add id
to list of bootstrap tools.
Submitted by: jhay

Only pre-build the "default" libcrypt, since it gets to set the symlinks
for the rest of the build. I'm not certain, but I think this determines
which crypt() goes into /sbin/init. This change shouldn't hurt anyway. :-)

Based on a suggestion by: bde

Urgent fix for bootstrap target. If the old symlinks are pointing to the
real source tree, the bootstrap target would attempt have cpio copy the
files over themselves, unlinking them first. I think this only happened
with make -DNOCLEAN world at the transition between a symlinked
objdir/tmp/usr/include/{sys,net,..} and real files.

Replace the partial symlink tree to src/sys/* in $OBJDIR/tmp/usr/include
with real copies. I'm sick of !@#&!^!@#*& mtree chowning directories in
my src/sys/* tree after it follows the symlinks. I still believe that
mtree is broken for doing this (introduced in mtree.c rev 1.5).

On the other hand, when /usr/bin/as is really a link to objformat, it's not
helpful to stop it running /usr/libexec/aout/as for real while bootstrapping.
Only force a strict path when we really have built all the tools in
$OBJDIR/tmp/usr/libexec/*.

Move objformat after binutils, otherwise we get chicken/egg problems when
building ELF. It may be that binutils and objformat need to both be moved
earlier in the list though, but that can wait till the dust settles.

Compile & use objformat as it was intended to, I have no idea why
that failed for me before :(

ELF preparation step 2:

Move a.out libraries to /usr/lib/aout to make space for ELF libs.
Make rtld usr /usr/lib/aout as default library path.
Make ldconfig reject /usr/lib as an a.out library path.
Fix various Makefiles for LIBDIR!=/usr/lib breakage.

This will after a make world & reboot give a system that no
longer uses /usr/lib/*, infact one could remove all the old
libraries there, they are not used anymore.

We are getting close to an ELF make world, but I'll let this
all settle for a week or two...

ELF preparation step 1:

Move our old a.out utils to /usr/libexec/aout.
Enable binutils and put the utils in /usr/libexec/elf
Enable objformat, a little helper program that calls the right
utils based on /etc/objformat and $OBJFORMAT.

This will enable the ELF generating tools.
Remember that this is only step one, the system is still compiled
and run in a.out format ONLY.

Problem left to solve: The BSD manpages wins over the GNU equivalents
as the are installed last. We need to distinguish between the manpages
somehow...

Make perl a build tool since it is required during a make depend.
This should fix the problem people have been having with perl/usub.

Now that FreeBSD/Alpha can be installed on a disk and run on its own,
a bootstrap build under NetBSD is an option.

This makefile will require further changes to ignore aout tools.

Don't use `&&' in any shell commands here. Using it to give conditional
execution is usually unnecessary in BSD Makefiles because BSD make
invokes shells with -e. Using it to give conditional execution is
often wrong in BSD makefiles because BSD make joins shell commands
when invoked in certain ways (in particular, as `make -jN'). Example
makefile:
---
clean:
cd /
false && true
rm -rf * # a dangerous command
---
This should terminate after the `false && true' command fails, but
it doesn't when the commands are joined (`false && true' is a non-
simple command, so -e doesn't cause termination). The b-maked version:
---
clean:
cd /
false; true
rm -rf * # a dangerous command
---
terminates after the `false' command fails (`false' is a simple
command, so -e causes termination). However, for versions of
make like gnu make that don't invoke shells with -e, this change
completely breaks the makefile.

This is one of the fixes for the bug suite that caused `make world'
to sometimes put raw cpp output in .depend files. Building of cc
sometimes failed, but the failure did not terminate the build
immediately, and various wrong versions of the cc components were
used until one was wrong enough to cause a fatal error.

Removed share/info from build-tools. It should never have been there,
and became redundant when it was put in SUBDIR, and became bogus when
WORLDTMP was introduced, and became broken when INFODIR was introduced.

PR: 6138

Fix the problem people are having building -current on a -stable system.
The headers that are installed in WORLDTMP are part of the interface
that includes libraries like libc, so they must be installed together.
This means that lib-tools and build-tools should be merged. The FreeBSD
build only works in hosted form where it is assumed that the installed
version contains adequate tools to build the latest release.

Added patch to build-tools (now needed to make gdb, sigh).

Pointed out by: andreas

Don't build dependencies unless NOCLEAN is set. This speeds up
`make world' by about 14% here (down to 4490 seconds real on a
K6/233). Temporarily skip this optimization when building with
-j, since there are still many broken makefiles.

Fixed NOCLEANDIR option. Cleaning of `.depend' was broken.

Put -nostdinc in CFLAGS, not in CC, and don't override the default
CC. This fixes enforcing use of ${WORLDTMP}/usr/include.

Don't install library man pages in ${WORLDTMP}.

Use more reasonable defaults for SUP variables (and print them
correctly) now that sup is long since dead and buried.

make includes has to execute "make beforeinstall" in src/lib/libcalendar
as well, otherwise "make depend" during "make buildworld" fails in
usr.bin/ncal

Don't build shared libraries twice for `make world'. This reduces
`make world' times by about 10%. Use a new `bootstrap-libraries'
target to build just enough (static) libraries to bootstrap the
tools, and delay building of shared libraries until all the tools
have been built.

Fixed `/usr/bin/make world'. The version of `make' that we buildis
nand use is named "make", not ${MAKE}.

Cosmetic: put '@' in front of a very long command.

Make the decision to compile kerberosIV a bit more secure by looking
for the appropriate directory.

Build the initial `make' in the temporary build tree so that its forced
cleaning doesn't affect the normal copy. Save a little time by not doing
unnecessary clean, depend nd cleandepend steps.

Reviewed by: mckay@freebsd.org

Added xargs to build-tools. It may be used for kdump and truss.

Restored an (apparently unnecessary) -B for installing lib-tools.
The install target is -j-unsafe in general, so we use -B for
installing everything in `make world'.

Don't force NOPIC for building tools at all for now, so that some
(slightly stale) shared libraries get installed before non-tools
are linked. The 2-stage build of the libraries has never really
worked, since we link to the libraries built in the first stage.

Don't force NOPIC or NOSHARED for building tools in the NOCLEAN case,
so that dirtier-than-previously tools aren't left lying around to be
installed.

Filter out NOPIC from ${MK_FLAGS} for `make ${OBJDIR}' for ld (and
some other things), so that the obj dir for ld/rtld always gets
re-made.

Fixed a literal "make".

Build all tools shared, and don't build any shared libraries for tools.

Build libraries (for linking the tools to) in a more correct order
(for linking freshly created shared libraries to each other). This
is probably a no-op now that shared libraries for tools aren't
built, but I didn't test any intermediate versions. Security-related
directories that are not built by default may now be misordered
for the shared case.

Don't build libcompat specially. It isn't used for tools, and
shouldn't be used in /usr/src (it is only used for IPXrouted and
crufty games).

Added missing ${.CURDIR} to existence tests for library directories.
Existence tests for top-level directories are still broken.

Test for library directories actually being built, not for directories
above them.

compile_et is now used early to build includes for libss, so build it
earlier. This is probably unnecessary.

Added now-necessary -B for installing headers for libss.

Removed now-unnecessary -B for building cleandepend for rpcgen.

Fixed missing SHLIBDIR in GCC_EXEC_PREFIX.

Add the path to the source .mk files to the beginning of .MAKEFLAGS
instead of to the end, so that there is more chance of it having
priority. Additions in /etc/make.conf still have priority for
sub-makes if they are also at the beginning, although this is
probably not wanted for building /usr/src.

Use ${MK_FLAGS} for building dependencies for tools. This saves
time building dependencies for tools that won't be created (mainly
profiled libraries) and will be necessary to give correct dependencies
when tools are built static.

Spell the object directory as ${OBJDIR} consistently (even when we know
that ${OBJDIR} == "obj").

Print ${DESTDIR} in messages where appropriate.

Fixed some misformattings.

Separated header creation from header installation in libss. Create
the libss headers before installing them in `make world'.

Change the bootstrap makefile inclusion to be general based on the
MACHINE. This lets me play with porting to m68k too. 8-) That is a
good test for portability because MACHINE != MACHINE_ARCH and m68k is
big endian.

MF22:
- After the install has been done, but before makedb is run, run
'ldconfig -R' to pick up new shlib dependencies.

Suggested by: Amancio and a whole lot of other folks

-DNOTOOLS to not rebuild any tools during buildworld

Test for ${MACHINE} == "alpha" and include the bootstrap makefile instead
of doing the normal build. When the alpha bootstrap is complete (there
is more work to do!), the alpha will build like the i386 does now.
I changed i386 references to ${MACHINE} now that we're multi-architecture.

Do not use -B in initial build of "make". It doesn't exist in 2.1.x.
Instead, list the targets one per command line.

Reviewed by: bde
Submitted by: bde (part)

Removed uneeded ${.MAKEFLAGS} again.
Originally committed in: rev.1.141
Spammed by: rev.1.149 (mouldy parallel make diffs)

Having to review all changes to this file _very_ carefully
led me to find 2 more errors which were there before. :)

Hooboy!

Did I ever spam this file good with that last commit. Despite 3
reviewers, we still managed to revoke the eBones fixes, TCL 8.0 support,
libvgl and a host of other new things from this file in the process of
parallelizing the Makefile. DOH! I think we need more pointy hats - this
particular incident is worthy of a small children's birthday party's worth of
pointy hats. ;-)

I certainly intend to take more care with the processing of aged diffs
in the future, even if it does mean reading through 20K's worth of them.
I might also be a bit more anal about asking for more up-to-date changes
before looking at them. ;)

Removed redundant 'all' arg to " Making Make" target.
Removed "-r RELENG_2_2" from 'update' target.

How the heck did this get back??

Changes to support full make parallelism (-j<n>) in the world
target.
Reviewed by: <many different folks>
Submitted by: Nickolay N. Dudorov" <nnd@nnd.itfs.nsk.su>

When bootstrapping lex, the object directory for lex/lib would be deleted
without being recreated. A read-only source compile would then fail shortly
afterwards trying to write .depend into the source tree.

Change for KTH KerberosIV.
-DMAKE_EBONES is changed to -DMAKE_KERBEROS4

When making make, do cleandir *before* objdir or we'll end
up with a blown objdir for the depend stage, leaving the
.depend file in ${SRCDIR} instead.

Backed out rev.1.136 and rev.1.144. Force SHARED=symlinks in a different
way - for buildworld only.

Rev.1.136 broke the (deprecated) external includes target. Rev.1.144 was
a half-baked backout of rev.1.136. It broke SHARED=symlinks for the
buildworld target and installation of subdirs of src/include for the
includes target.

When calling the src/include Makefile use the shorthand ``all beforeinstall''
inplace of ``all installhdrs symlinks'' so those of us who run with
SHARED=copies get the right results.

Back out use of -m, since 2.1.x apparently do not have it. (sigh..)

Fix the ${BINFORMAT} definition problem during 'make {build}world'. The
wrong switches were being passed to 'make' that caused sys.mk to come from
the /usr/share/mk location but the bsd.*.mk files to come from the
source tree in src/share/mk. This was bound to break sooner or later.

Submitted by: Dmitrij Tejblum <dima@tejblum.dnttm.rssi.ru>

Remove unneeded ${.MAKEFLAGS} being explicitly passed on via the command
line..

PR: 4368

Use new installhdrs target to install tcl includes.

Added directories in the includes target so that it installs
everything that a full install would (modulo bugs in rpcsvc/Makefile).
The missing things weren't used (except for mp.h from libmp), but
there are relatively few of them and it's easier to be complete.

Restored rev.1.122, which was clobbered by copying from -stable in
rev.1.134.

Split long lines.

Pass command line args to 'buildworld' and 'installworld' etc. Otherwise
'make -DNOCLEAN world' is completely ignored.

In the includes target, call the header install target explicitly and then
call the 'symlinks' target to build the /usr/obj/tmp/install/sys tree since
In the includes target, call the header install target explicitly and then
call the 'symlinks' target to build the /usr/obj/tmp/install/sys tree since
we set this up manually earlier, and if we do a 'make all install', we
replace the symlink tree in the obj dirs with new copies of the files
which breaks any chance of -DNOCLEAN working.

fix a make world problem.
updated beforeinstall target.
mp.h from gnu/lib/libgmp.h has to be installed to /usr/include
so that usr.bin/chkey and usr.bin/newkey can be compiled.

Merge from -stable; NOTCL hook in beforeinstall.

Merge buildworld/installworld patch to Makefile from RELENG_2_2. Patch
a couple *.mk files to enable -current world building on really old
machines (e.g., 2.1.5).

Reviewed by: too many many people to list here, special thanks to bde

More whitespace/tab fixes.

Submitted by: bde

Consistently use tabs in the leading comments.

Submitted by: bde (yes, I'm serious)

Remove sgml stuff from the build-tools target. It is no longer used
in a make world.

Add libmp and libz to the includes target. This makes "make includes -DCLOBBER;
make depend; make all" work. I think there should be a better than adding
each library directory here.

make beforeinstall in libalias from the includes target

Fix comment about LOCAL_DIRS. Now it works as advertised. :)

Remove evil MAKE_LOCAL and MAKE_PORTS hacks and replace them with
a simpler, more general LOCAL_DIRS hack.
Reviewed by: asami

Forgot to commit this.. Do a 'make all' in src/include before trying
to install it. Otherwise osreldate.h breaks.

Pointed out by: Doug Rabson <dfr@nlsystems.com>

Use `mk' files in share/mk (relative to the top level of the source
tree) instead of the system default ones in /usr/share/mk. This
only works for makes run from the top level, and a few not too out
of date system makefiles must be present to start up. Bootstraps
from can still be done from scratch by giving the path to an up to
date share/mk on the command line.

Removed bootstrap `mk' target and use of it. The bootstrap didn't
actually work for the non-null DESTDIR case.

Change default build to include libc_r, but allow disadvantaged souls
to turn off the build.

Bootstrap the info `dir' file from here.

Process "lib/csu/i386" at the beginning of the "libraries" target,
instead of in the middle of it. Soon, "c++rt0.o" will be linked
into all shared libraries. An up-to-date version of it must be in
place before any shared libraries are built.

This is one piece of the solution for PR gnu/3035 (gcc -shared).

Back out csu fix, because it was just fixed differently

For csu/i386 add "all" before "beforeinstall", because "beforeinstall"
wants to install "all" targets which not exist at this stage

Revert $FreeBSD$ to $Id$