Copy stable/11@r303970 to releng/11.0 as part of the 11.0-RELEASE
cycle.

Prune svn:mergeinfo from the new branch, and rename it to RC1.

Update __FreeBSD_version.

Use the quarterly branch for the default FreeBSD.conf pkg(8) repo and
the dvd1.iso packages population.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

mkimg: Indicate that input file pages are unlikely to be reused.

mkimg(1) uses a swap file to back input file chunks. When the output file
is being written out, blocks of the swap file are mapped and their contents
copied. This causes the backing VM pages to enter the active queue, and when
the output file is large relative to system memory (as is generally the
case), can result in a shortfall of inactive memory. This causes the
pagedaemon to aggressively scan the active queue and swap out process
memory in an attempt to meet the shortfall. Because mkimg's input files
are typically the intermediate result of some build process, there's no
need to push them all through the active queue. Use madvise(2) to indicate
that the backing pages may be reclaimed in preference to active pages. In
the case of the swap file, these pages will be freed as soon as mkimg
exits anyway.

When using mkimg on a desktop-class system with large amounts of dirty
process memory, this change substantially improves mkimg runtime and
reduces swap usage.

Reviewed by: marcel
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D6654

Make image_copyout_zeroes() an interface function.

SEEK_DATA has interesting behaviour for sparse files on ZFS. A sparse file
with 128K of random data and truncated to 800K can have SEEK_DATA return -1
when given an offset of 128K. On UFS, the SEEK_DATA returns 800K (the size
of the file). SEEK_HOLE on ZFS seems to behave the same as UFS.

To handle this, map -1 to the size of the file (`end') when lseek returns
this for either SEEK_HOLE or SEEK_DATA. When sparse files are not supported
by the file system both `hole' and `data' will now be equal to `end' and we
will treat the entire file as data. This way, the -1 return for SEEK_DATA
on ZFS will end up doing the right thing.

Reported by: gjb@

MFC after: 3 days

Improve performance of mking(1) by keeping a list of "chunks" in memory,
that keeps track of a particular region of the image. In particular the
image_data() function needs to return to the caller whether a region
contains data or is all zeroes. This required reading the region from
the temporary file and comparing the bytes. When image_data() is used
multiple times for the same region, this will get painful fast.

With a chunk describing a region of the image, we now also have a way
to refer to the image provided on the command line. This means we don't
need to copy the image into a temporary file. We just keep track of the
file descriptor and offset within the source file on a per-chunk basis.

For streams (pipes, sockets, fifos, etc) we now use the temporary file
as a swap file. We read from the input file and create a chunk of type
"zeroes" for each sequence of zeroes that's a multiple of the sector
size. Otherwise, we allocte from the swap file, mmap(2) it, read into
the mmap(2)'d memory and create a chunk representing data.

For regular files, we use SEEK_HOLE and SEEK_DATA to handle sparse files
eficiently and create a chunk of type zeroes for holes and a chunk of
type data for data regions. For data regions, we still compare the bytes
we read to handle differences between a file system's block size and our
sector size.

After reading all files, image_write() is used by schemes to scribble in
the reserved sectors. Since this never amounts to much, keep this data
in memory in chunks of exactly 1 sector.

The output image is created by looking using the chunk list to find the
data and write it out to the output file. For chunks of type "zeroes"
we prefer to seek, but fall back to writing zeroes to handle pipes.
For chunks of type "file" and "memoty" we simply write.

The net effect of this is that for reasonably large images the execution
time drops from 1-2 minutes to 10-20 seconds. A typical speedup is about
5 to 8 times, depending on partition sizes, output format whether in
input files are sparse or not.

Bump version to 20141001.

Add image_data() for checking whether a sequence of blocks has data.
Use this for VHD and VMDK to avoid allocating space in the image
for empty sectors.

Note that this negatively affects performance because mkimg uses a
temporary file for the intermediate storage. When mkimg has better
internal book keeping, performance can be significantly improved.

Add VHD support to mkimg(1). VHD is used by Xen and Microsoft's Hyper-V
among others.

Add an undocumented option for unit testing (-y). When given, the image
will have UUIDs and timestamps synthesized in a way that gives identical
results across runs. As such, UUIDs stop being unique, globally or
otherwise.

VHD support requested by: gjb@

Create our temporary file in $TMPDIR, if the environment variable
is set. /tmp otherwise.

Submitted by: Dan McGregor <danismostlikely@gmail.com>

Fix CID 1215124: Handle errors properly.

Fix CID 1215129: move the call to lseek(2) before the call to malloc(3)
so that the error path (taken due to lseek(2) failing) isn't leaking
memory.

MFuser/marcel/mkimg:
Add support for different output formats:
1. The output file that was previously written is now called the raw format.
2. Add the vmdk output format to create VMDK images.

When the format is not given, the raw output format is assumed.

Compile on x86 (ILP32).

Add image_get_size() so formats can query about the image size.
Create the VMDK (embedded) descriptor file and round its size
to a multiple of 512.

TODO: We need the name of the output file -- it's in the extent
description. For now, just use "mkimg.vmdk".

Have image_write() call sparse_write() so that we can use SEEK_HOLE
and SEEK_DATA to "quickly" find occupied sectors. This is all short-
lived, because the image should not be kept in a file, but in memory.
To be precise: the image API should be both efficient and scalable
and using a file is not efficient -- it works, which is what I need
right now.

When write sparse files, make sure to truncate(2) the file. Otherwise
any hole at the end is lost and the file is not of the right size.

Provide a file-based implementation for the image API. This almost
copies the logic verbatim from mkimg.c to image.c. The difference
is that in image.c the temporary file is always created, whereas
before we only created a temporary file when writing to stdout.

Cleanup mkimg.c now that the song and dance of using a temporary
file is gone.

Switch to the image API:
1. Replace calls to mkimg_set_size() with calls to image_set_size()
2. Remove the mkimg_set_size() function
3. As above but for mkimg_write() and image_write()

Note that this breaks mkimg(1). The image API has no implementation.
Hence doing it on my branch :-)

Add image.c and image.h. These files will contain functions to implement
the management of the raw image that's being created. The most notable of
the API is that there's no file descriptor argument. This is because the
image is managed in memory.

Once we have the in-memory (raw) image, we can write it out acording to
different formats. This two-pass approach shields schemes from formats
and formats from schemes.