MFC r309478:

DIRDEPS_BUILD: Fix '/' in group names creating a bad cookie.

MFC r309477:

Revert r296585.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Merge the projects/release-pkg branch to head.

This allows packaging the base system with pkg(8), including
but not limited to providing the ability to provide upstream
binary update possibilities for non-tier-1 architectures.

This merge is a requirement of the 11.0-RELEASE, and as such,
thank you to everyone that has tested the project branch.

Documentation in build(7) etc. is still somewhat sparse, but
updates to those parts will follow.

Sponsored by: The FreeBSD Foundation

These group names may be used as a cookie, so replace any non-fs-safe characters.

One example is in cddl/usr.sbin/dtrace/tests/common/aggs. It could be
fixed but other uses of this would break, especially in the
DIRDEPS_BUILD which uses the group names for stage cookies.

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division

DIRDEPS_BUILD: Fix some things building in MAKELEVEL0.

For PROGS this was recursing twice since MAKELEVEL0 is for 'dirdeps'
which then really builds in a sub-make.

Sponsored by: EMC / Isilon Storage Division

Fix style and remove excess / for installing SYMLINKS.

Sponsored by: EMC / Isilon Storage Division

Stop hiding link install commands.

This is no longer needed now that a .for loop is used rather
than inline shell script.

Sponsored by: EMC / Isilon Storage Division

Make installing to a non-existent directory an error.

Before this, if a file was installed to DESTDIR/some/dir and that directory
was missing due to not having ran 'make distrib-dirs' yet, the file would
be installed as 'some/dir'. For something like bsd.incs.mk with INCLUDEDIR
being a sub-directory of /usr/include, this could result in all of the headers
being installed to a file rather than getting a directory of them.

Now it will error that the file/directory does not exist rather than hide
the issue.

Another option being discussed is to implement GNU's install -D flag which
would auto create any missing directories.

This is a mitigation of the problem. The proper order to the build is to
run 'make distrib-dirs' first, but that can be forgotten if building from
a sub-directory after updating the source code to the latest revision.

MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division

Revert r289282 for now as the interaction with a directory containing
bsd.files.mk and bsd.subdir.mk is recursing too many times.

Replace the out-of-place includes/files/config handling in bsd.subdir.mk with
more typical ALL_SUBDIR_TARGETS entries and target hooks in bsd.incs.mk,
bsd.files.mk and bsd.confs.mk.

This allows the targets to be NOPs if unneeded and still work with the
shortcut 'make includes' to build and then install in a parallel-safe manner.

Sort and re-indent the ALL_SUBDIR_TARGETS with the new entries.

Sponsored by: EMC / Isilon Storage Division

Correct a comment in bsd.incs.mk forgotten in r274662 and copied into bsd.confs.mk.

The bsd.confs.mk may be wrong but for now fix it.

Sponsored by: EMC / Isilon Storage Division

In jobs mode we can use .ORDER to force stage_links to run after other
stage_* targets.
In non-jobs mode we can achieve the same result by simply introducing
the targets in the correct order.
Thus in bsd*.mk we simply add targets to STAGE_TARGETS which we
realize in meta.stage.mk

Reviewed by: bdrewery

META_MODE: Fix staging not respecting _DIR overrides.

This fixes atf-c.h not properly being installed to /usr/include/ (in
the stagedir) via its override of 'INCSDIR_atf-c.h= ${INCLUDEDIR}'.

This fixes building things that depend on atf.

Staging seems to ignore OWN/GRP/MODE settings and needs further exploration.

Sponsored by: EMC / Isilon Storage Division

Simplify symlinks on include files by using bmake's multivariable .for loops

Add META_MODE support.

Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision: D2796
Reviewed by: brooks imp

Rework PRIVATELIB

Now when a lib is marked as PRIVATELIB it is renamed into libprivate$foo instead
of being installed in /usr/lib/private and playing with rpath.

Also allow to install headers for PRIVATELIBS in that case the headers will be
installed in /usr/include/private/$foo

Keep the headers under a private namespace to prevent third party build system
to easily find them to ensure they are only used on purpose.

This allows for non base applications to statically link against a library in
base which is linked to a privatelib

Treating PRIVATELIBS as regular libraries allows to push them into our current
compatX packages if needed.

While here finish promotion of libevent as PRIVATELIB
Install header for bsdstat and libucl

Differential Revision: https://reviews.freebsd.org/D2365
Reviewed by: brooks, des
Discussed with: imp

Acutally, replaces NO_INCS with WITHOUT_INCLUDES and make
WITHOUT_TOOLCHAIN imply it instead.

Sponsored by: Netflix

Eliminate NO_INCS. It is used one place, and MK_TOOLCHAIN=no will
handle the job just as well. Opt for simplicity rather than one more
option.

Replace all known uses of ln in the build process with appropriate
install -l invocations via new INSTALL_LINK and INSTALL_SYMLINK
variables.

Sponsored by: DARPA, AFRL
Reviewed by: ian, ray, rpaulo

Reimplementation of world/kernel build options. For details, see:

http://lists.freebsd.org/pipermail/freebsd-current/2006-March/061725.html

The src.conf(5) manpage is to follow in a few days.

Brought to you by: imp, jhb, kris, phk, ru (all bugs are mine)

Handle NO_INCS solely inside bsd.incs.mk.

Don't install includes if NO_TOOLCHAIN

Make sure the only thing that follows .endif or .else is a comment.

Try really hard to fix parallel installs. Add a bunch of .ORDER
directives to ensure that all realinstall sub-tasks are executed
after beforeinstall, similarly ensure that all afterinstall sub-
tasks are executed after realinstall. Demonstration:

all: task1 task2
.ORDER: task1 task2

task2: task2_subtask
.ORDER: task1 task2_subtask

task1 task2 task2_subtask:
@sleep `jot -r 1 0 1.0`
@echo ${.TARGET}

Without the second .ORDER directive, task2_subtask can be run in
parallel with task1.

Spotted by: Andrea Campi <andrea@webcom.it>

Rename `includes' to `buildincludes'.
Rename `incsinstall' to `installincludes'.
Make `includes' a -j safe shortcut for `buildincludes' + `installincludes'.
`buildincludes' and `installincludes' are SUBDIR friendly, if run directly.

Added new bsd.incs.mk which handles installing of header files
via INCS. Implemented INCSLINKS (equivalent to SYMLINKS) to
handle symlinking include files. Allow for multiple groups of
include files to be installed, with the powerful INCSGROUPS knob.
Documentation to follow.

Added standard `includes' and `incsinstall' targets, use them
in Makefile.inc1. Headers from the following makefiles were
not installed before (during `includes' in Makefile.inc1):

kerberos5/lib/libtelnet/Makefile
lib/libbz2/Makefile
lib/libdevinfo/Makefile
lib/libform/Makefile
lib/libisc/Makefile
lib/libmenu/Makefile
lib/libmilter/Makefile
lib/libpanel/Makefile

Replaced all `beforeinstall' targets for installing includes
with the INCS stuff.

Renamed INCDIR to INCSDIR, for consistency with FILES and SCRIPTS,
and for compatibility with NetBSD. Similarly for INCOWN, INCGRP,
and INCMODE.

Consistently use INCLUDEDIR instead of /usr/include.

gnu/lib/libstdc++/Makefile and gnu/lib/libsupc++/Makefile changes
were only lightly tested due to the missing contrib/libstdc++-v3.
I fully tested the pre-WIP_GCC31 version of this patch with the
contrib/libstdc++.295 stuff.

These changes have been tested on i386 with the -DNO_WERROR "make
world" and "make release".