Copy stable/10@r272459 to releng/10.1 as part of
the 10.1-RELEASE process.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

MFC r262499, r262505, r262507, r262509:

r262499:
release.sh:
- Add a VCSCMD variable that defaults to 'svn checkout',
and update places 'svn co' is used directly.
- After sourcing a configuration file, prefix SRCBRANCH,
PORTBRANCH, and DOCBRANCH with the SVNROOT.
- Properly capitalize 'FreeBSD.org' in the default SVNROOT.
- Update Copyright.

release.conf.sample:
- Add an example to use git instead of svn, by nullifying
SVNROOT, and setting SRCBRANCH, DOCBRANCH, and PORTBRANCH
to the URL fo a git repository.

release.7:
- Document VCSCMD.

r262505:
In release/Makefile, remove exclusion of CVS directories in the
src/ and ports/ distributions.

While I am thinking about it, exclude .git directories for src/
and ports/, as somewhat of a followup to r262499.

r262507:
Chase r262505, and remove CVS exclusion from picobsd builds.

r262509:
Rework how WORLD_FLAGS and KERNEL_FLAGS are set, to remove an
unnecessary expr(1) call.

Sponsored by: The FreeBSD Foundation

Copy head (r256279) to stable/10 as part of the 10.0-RELEASE cycle.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

comment out some stale loader configurations.

generate multiple host keys and do that unconditionally

r253616 nuked BINMAKE so we need to adapt to the new definition

use a plain 'make toolchain' to build toolchain, and
support parallel make on this stage.

remove a leftover comment

enable building picobsd with CC instead of CLANG

use MALLOC_PRODUCTION for picobsd builds, be more frugal in memory usage.

Omit clang when building picobsd, it takes way too long.
Also note that loader.conf support is incomplete.

now picobsd cross builds work (tried with host amd64, target i386 ).
The fix involved adding a proper build of ld-elf.so.1 ,
and also replacing ldd with objdump (suggested by Garrett Cooper)
to build the list of shared libraries needed by the binaries
and libraries on the target.

remove some stale options (such as running without /boot/loader)

improve support for multi-arch and cross-arch builds, by adding
a suffix to the kernel config file and build_directory.
(cross builds not clean yet, a cross-built kernel boots
but fails when starting /sbin/init)

some tweaks to build picobsd for different architectures.

following the recent (218130) changes to Makefile and Makefile.inc1,
we need to set TARGET and TARGET_ARCH to get a correct WMAKEENV.
I am setting both to i386 since this is what picobsd is used for,
though there might be a better fix.

Add initial support for parallel make. This is disabled right now,
because there are incorrect dependencies somewhere which require
to run picobsd 2-3 times to complete a build.

MFC after: 2 weeks

move the initialization of BINMAKE earler, so it is
available throughout the entire build process.

Submitted by: Matteo Landi
MFC after: 3 days

use make from the toolchain, as the source tree may depend on
specific features (e.g. on recent HEAD, rtld fails with older 'make')

MFC after: 3 days

use log instead of logverbose in three places so the build won't need
input to run to completion

MFC after: 3 days

clear leftover from previous computations

more support to import files and libraries from the host.

fix a typo (not that it matters)

when calling ldd, use the cross libraries and not the host version

MFC after: 3 days

fix wrong name for the iso!

Add code to generate a bootable ISO image, so we are finally
free from the 2.88MB that we had using El Torito emulation.

The --iso option was already there, just didn't do anything before.

Submitted by: Marta Carbone
MFC after: 3 days

add support for shared "crunch" binary.

add a function to help copying shared binaries from the output
of a buildworld.

remove unused "package" entry.
default to use devfs (we do that anyways)

Many small changes in comments

add the option to picobsd to copy files from the host filesystem
without root privs. This is done, among other things, replacing
the absolute paths in the symlinks with relative paths, so we
do not need to do a chroot to follow them.

Still need to update the manpage.

MFC after: 3 days

when initializing, also build _includes and _libraries

On passing, fix a wrong comment

MFC after: 3 days

use /boot/loader by default, because the boot code seems to
have problems with kernels larger than 4MB.
Add a flag to avoid the /boot/loader and use the old method.

Add support for an additional makefile to perform custom manipulation
(this is not documented yet).

Add support for building an ISO image (not complete)

Enable GCC stack protection (aka Propolice) for userland:
- It is opt-out for now so as to give it maximum testing, but it may be
turned opt-in for stable branches depending on the consensus. You
can turn it off with WITHOUT_SSP.
- WITHOUT_SSP was previously used to disable the build of GNU libssp.
It is harmless to steal the knob as SSP symbols have been provided
by libc for a long time, GNU libssp should not have been much used.
- SSP is disabled in a few corners such as system bootstrap programs
(sys/boot), process bootstrap code (rtld, csu) and SSP symbols themselves.
- It should be safe to use -fstack-protector-all to build world, however
libc will be automatically downgraded to -fstack-protector because it
breaks rtld otherwise.
- This option is unavailable on ia64.

Enable GCC stack protection (aka Propolice) for kernel:
- It is opt-out for now so as to give it maximum testing.
- Do not compile your kernel with -fstack-protector-all, it won't work.

Submitted by: Jeremie Le Hen <jeremie@le-hen.org>

Fix build of the "package" target, ignoring non existing
floppy types (in fact, we have only one left!)
Also cleanup some code, using || and && in some places, and
using "export VAR=value" instead of two separate lines.

Fix picobsd builds. Changes include:
- use proper make configuration for the build, using
${BINMAKE} as evaluated by ${VERSION}/src/Makefile
- remove -lmytinfo from crunch.conf
- remove support for login_access in tinyware/login
- remove "machine i386" from the kernel config file

It might actually be interesting to extend the script to do
a cross build for !i386 ...

MFC after: 3 days

Send the pcvt(4) driver off to retirement.

unbreak the build script.
Instead of using vnconfig/mdconfig it relies on the makefs
port to build the image (UFS1) without need for root permissions.

NOCLEAN -> NO_CLEAN
NOCLEANDIR -> NO_CLEANDIR

NOSHARE -> NO_SHARE

NOHTML -> NO_HTML

NOGAMES -> NO_GAMES

NOFSCHG -> NO_FSCHG

NODOCCOMPRESS -> NO_DOCCOMPRESS
NOINFO -> NO_INFO
NOINFOCOMPRESS -> NO_INFOCOMPRESS
NOLINT -> NO_LINT
NOPIC -> NO_PIC
NOPROFILE -> NO_PROFILE

NOLIBC_R -> NO_LIBC_R
NOLIBPTHREAD -> NO_LIBPTHREAD
NOLIBTHR -> NO_LIBTHR

Start the dreaded NOFOO -> NO_FOO conversion.

OK'ed by: core

Fix various issues with the picobsd build script in -current:

+ declare some shell variables as 'local'
+ add a missing ';;' in a 'case' statement
+ centralize newfs options
+ properly override GID and UID when installing include files and libraries
+ add a missing '-r' in disklabel commands (maybe not necessary, but it
does not harm and it was used in the RELENG_4 version)
+ no more require 'perl' when installing the secondary loader
+ use '-t rsa1' when invoking ssh-keygen

Remove obsolete disklabel and newfs options.

Replace a logverbose call with log

Enable building of picobsd using CURRENT sources again.

Following a suggestion by Ruslan, the initial creation of the
includes and libraries (and build tools) is now done by
invoking "make buildworld" (with -DPICOBSD which eventually will
limit the amount of stuff built with a 2-line change in Makefile.inc1).
The correct environment is then used for subsequent builds.

Also remove write_mfs_in_kernel.c in favour of using dd

All the above is conditional on __FreeBSD_version, as the previous
method still worked for versions earlier than 500035, and I am
unsure on how the "new" method works for earlier versions.

Finally, note that the crunch.conf files need some work because
some libraries (e.g. gmd) have gone away from the base installation.

DESTDIR does not anymore does the -I${DESTDIR}/usr/include magic
in <bsd.prog.mk> and <bsd.lib.mk>.

Reviewed by: luigi

Gracefully exit when no MFS signature is found in the kernel.

Add support for a new "import_files" option (see manpage for details).

Fix detection of existing ssh host key, generate the new one
in floppy.tree instead of mfs_tree.

Do not cleanup ${BUILDDIR}/floppy.tree, it might be useful to look
at it after an image has been built.

Misc fixes to the build script:

+ fix some dialog entries to correctly modify variables instead of working
in a subshell

+ add a logverbose function for debugging purposes

+ force 512/4096 blocks on filesystems

+ use 'auto' for disklabel so it works irrespective of the floppy size.
This is useful for larger images than 1720k

Use { } instead of ( ) for grouping shell commands so that
variable assignments are retained ( () creates a subshell)

Reported-by: Michael Bretterklieber <mbretter@inode.at>

Whoops... left some junk in previous commit.

Let "make" use the .mk files from the correct source tree instead
of the default ones (which could be incorrect when doing cross builds).

Also, try to be backward compatible when compiling wmk (the goal being
able to use the most recent "picobsd" script on a wide range of
source trees).

Use a local write_mfs_in_kernel.c instead of the one in ncvs/src/release
which is likely to be removed shortly, after repo-copying it from
there.

Provide a new picobsd option, "--modules", to also build modules
(modload and friends are sufficiently small to consider their inclusion
in PicoBSD images).

Use host keys from the config directory if they exist, instead of
regenerating them.

Make the script handle relative paths for the source directory.

A somewhat large change of this script.

From the user's perspective:
* everything is now built outside the source tree (more precisely,
in `pwd`/builddir-${name}/ ) except for the kernel config file(s)
which still need to be copied into src/sys/i386/conf because of
"config" limitations. I am not sure if there is an easy way
to get away from this without changing "config" or replicating
some part of the source tree.
This is really the only change that most users should worry about,
but it is a good one.

* if you do cross-compiles (using "picobsd --src somedir/src [--init] ... ")
then the libraries and include directories etc. are searched/created
in "somedir/usr" ;

* you can do most things (basically build the kernel and the crunched
binary and the filesystem trees) without root privileges. You need
privileges to use mdconfig/vnconfig to create the actual MFS and
floppy image, unfortunately.

* the -v option now prints some diagnostic but does not stop for
user input at each step. You need to specify -v -v to have the
old behaviour.

Internally, the script has been reshuffled quite a bit to support
the above features. Many shell variables have been renamed or
made local in an effort to avoid undesired side effects. There is
a somewhat better error handling in case something goes wrong.

Move a bunch of initialization into a function which must be run
before parsing the command line.
Move code to build include and libraries in a separate function,
so we can use the verbose flag for that.
Chang ownership of some directories so more of the build process
(namely, builds of include and libs) can be run without root
permission (we still depend on root permission to mount a memory
filesystem).

On -current with devfs there is no MAKEDEV in /dev anymore, so use
it from ${SRC}/etc/MAKEDEV instead.

We may want to do this on -stable as well, but for now -stable still
uses /dev/MAKEDEV.

Relocate the memory disk selection code to the same place the other
variables are set.

Make the script aware of mdconfig, so it can hopefully run on -CURRENT
as well. This works by selecting "md" or "vn" depending on "uname -r"
output, so we can use the same script on -CURRENT and -STABLE.
Also included minor bugfixes and code cleanup.

Testers welcome, as this code has only been tested on -STABLE
(and for this reason I am doing an immediate MFC).

Move commands to create stripped kernel into Makefile.conf,
so as to make the "picobsd" script less version-specific.

Improve handling of cross-builds (which requires creation of
includes and libraries for the new source tree).

The "picobsd" script will not probably work on -current because it still
uses 'vn' instead of 'md', but i am commiting it anyways to keep it
in sync with the version in -stable.

Supply a default value for the kernel config binary.

Bugfix: Use the correct variable for picobsd type (same bug is
present in STABLE). Also add a bit more logging (when enabled).

Move handling of crunch* files into Makefile.conf, which is a
better place to handle dependencies.

Make another step at helping cross-compiling: when the user specifies
an alternate source tree, the script takes care of creating include
files and libraries for the new tree.
Furthermore, build and use a version of the "config" program which
matches the new sources.

It takes a long time to create libraries, and it might even not do
the right thing at once, there might be some dependencies that i
have forgotten. At any rate, with this code i have been able to
build a working picobsd image using -CURRENT sources on -STABLE

MFC after: 3 days

Move variable assignement after argument parsing, to improve handling
of alternate source trees.

Also improve verbose behaviour.

Sync picobsd script with the one in -stable, and also
patch boot2 so it goes to load /kernel right away

Perl code from Josef Karthauser

remove a reference to a nonexisting script, "clean", when
building all images. Also, build "bridge" image as well.

Reported-by: Bruce Montague

MF4S: Remove some unused error messages. (revision 1.1.2.5)

Bring the new script into -current. Still untested here, but there
is a high chance that they work modulo minor things.