Add new sets: base32, debug32, manhtml

- base32 contains (when MKCOMPAT=yes) shared libraries for 32-bit
compatibility, previously included in base

- debug32 contains (when MKCOMPAT=yes) debug symbols and static libraries
containing debug symbols for 32-bit compatiblity, previously included
in debug

- manhtml contains (when MKHTML=yes) the HTML files previously included
in 'man', which are of limited utility without third-party software.

The motivation for this change is to be able to easily exclude sets
from CD-ROM images that go over the size limit without xz compression
(which many NetBSD platforms struggle to extract at acceptable speeds).

Quote argument to -n as required. [ -n ] is not a syntax error but
evaluates to true.

PR/57925: Jan-Benedict Glaw: Honor $MKREPRO_TIMESTAMP, don't embed name and
stamp in gzip.

evbppc: Add initial support for the Nintendo Wii

branches: 1.78.2;
Move DRM-driver firmware from base to its own set, gpufw

This set is only installed on amd64,i386,evbarm.
This set is installed on minimal installs and on install media, in
case someone needs it for basic driver functionality.

Comments:
Switched to a single MK tunable for it - that is probably unneeded.

An upcoming DRM update will include even fatter firmware, and we'd
like to minimize the impact of it.

Fix fallout from mkimage rev 1.76.

For mkimage:

- Update "size" if auto-calculated.
- Use "dd bs=1" instead of non-portable "head -c".
- Some style nits.

For MD make_label() functions:

- Stop using "newsize" as image size in MB, use "size" instead.

embedded/mkimage: Pad image with zeros to multiple of 4 MB plus 1 MB.

Otherwise, there may not be enough space after the ffs partition for a
gpt, leading to very confusing results.

Create an "octeon.img.gz" bootable image for evbmips64-eb release builds.

Add support for hybrid MBR/GPT images.

Add GPT support to mkimage.

Separate devicetree .dts -> .dtb building from kernel builds. They are now
part of a separate set, "dtb.tgz", and only built when MKDTB=yes. This
defaults to yes for earmv[67]* and aarch64, and no everywhere else.

Derive the FAT type from the MBR partition type and pass it on to makefs
when crating a msdos file system.

Split out /rescue to its own set and adapt installers/images to add it.

This is meant to make updates safer: if something goes wrong with updating
base, we still have the old, standalone /rescue to recover from.

branches: 1.69.4; 1.69.6;

Be more precise about exactly what fails when something does.

Relying upon set -e to abort things is sort of OK (it is not
a recommended option to use in general - too many odd special cases),
but only if user can work out from the "build failed" what actually
went wrong.

Tested only on amd64 build (for this, i386 is the same) - if anyone
has problems on builds for other systems, please let me know. However
the changes affect only failure paths, the most likely problem would
be for a build to fail to halt on an error, and I hope I have avoided
that. There should be no difference at all to error-free builds.

Handle use of TMPDIR in cleanup as well.

Honour ${TMPDIR} if set for location of image, rather than always
simply using /tmp

Replace HUMMINGBIRD_A31 with SUNXI kernel on armv7.img and include .dtb
files for SUNXI and TEGRA kernels on the MSDOS partition.

add the tests set.

branches: 1.64.4;
Now that FAT+FFS images include the BSD partition in the MBR partition
table, install the disklabel after setting up MBR partitions. This moves
the disklabel to the BSD partition from the start of the disk, avoiding
a conflict with the ODROID-C1 bootloader.

Improvements and fixes for FAT+FFS images:
- Add a "NETBSD" volume label to the MSDOS file-system.
- Explicitly initialize boot sector.
- Correct an issue where the MSDOS file-system was larger than the
partition table entry.
- Add NetBSD partition to the MBR partition table.

Mark the msdos parition active for boards like the BeagleBone Black

branches: 1.61.2; 1.61.4;
use a larger (64KB) block size, this helps SD card performance

Add a new endianes flag to mkimage and pass it on big endian arm builds.
From Rin Okuyama.

fix mkubootimage

Add some documentation:
Tell what this does, and what it expects from conf files

Properly spell X' name. From X(7):

The X.Org Foundation requests that the following names be used when
referring to this software:

X
X Window System
X Version 11
X Window System, Version 11
X11

initialize resize before pulling in board config

Add support for auto-growing the root partition. Enable it for rpi.img.

resizing of ffsv2 still doesn't work well, so revert previous, and instead
explicitly set fsize=2048/bsize=16384, which gives a pretty decent boost
to resize_ffs performance on its own

Use FFSv2 instead of FFSv1 for the root file-system. This should fix the
"resize_ffs takes forever" issues with evbarm images -- growing a v2
file-system is much faster than v1.

Fix previous so that the new generated rc.conf is picked up and that
any new new mount points (e.g. /proc and /kern) are generated.

While here increate the fs size so that postfix can actually build
/etc/mail/aliases.db

PR/49596: mkimage: incorrect permissions on evbarm images

Use NetBSD.dist in our mtree specification to populate the top level
directories, etc.

For each generated file hand craft an mtree entry.

Remove specialdirs as NetBSD.dist creates /proc and /kern.

Use [dx]sets_p as flag for debug/X sets inclusion. xsets is the list of
sets.

fix image building:
- round up the total size of the disk image
- fix fdisk offset for netbsd ffs images

branches: 1.48.4;
now that fdisk has optional arguments, don't pass -1

provide a -r rootdev option to handle the wd/sd/ld lossage.

these changes produce a bootable system under qemu, but its root is trashed.

Add preliminary support for creating x86 usb sticks with mkimage.
This is untested, and just a checkpoint of the work. Image looks
ok, but I have not tried if it boots.

Fix build release of evbarm which had been failing in mkimage
beagleboard_smp with the elusive message:

"gzip: can't stat: gzip: No such file or directory"

This was due to GZIP=gzip being defined in mkimage; we now use:

GZIP_CMD=${TOOL_GZIP:-gzip} # ${GZIP} is special to gzip(1)

as distrib/sets/sets.subr already does in these files as well:

distrib/utils/embedded/mkimage
distrib/utils/embedded/conf/beagleboard.conf
distrib/utils/embedded/conf/rpi.conf
distrib/utils/embedded/conf/rpi_inst.conf

SSD cards is stupid.

explain -m better

tidy up, fix usage

Use TOOL_GZIP in mkimage, and pass the TOOL_GZIP into it.

- explanations for single letter flags
- flag to add debug sets
- flag to minimize writes to sd's

Double extra to fix image size to fix build.

For ffs, set bytes per inode to 4K. Otherwise we get a density based on
the contents of the image, which aren't very useful after the file-system
has been resized -- for rpi.img, this was 22K per inode!

Add a new installer image for Raspberry PI to evbarm

Includes the following major changes:

1) Add new configmenu item to sysinst to allow creation of users at
post-install time.
2) Add an sshramdisk type to evbarm, which is a ramdisk with sshd
enabled, allowing users to install on a headless PI by sshing to the
device with the account "sysinst" password "netbsd" to begin the
installation. (Note, neither the account, nor the ssh setup are copied to
the finalized installation image)
3) Change mkimage to build a boot-only image, without the root filesystems.

Much of the sshramdisk code could be reused on other ports easily.

don't use [ -f $i ] && cat $i, since we are using -e

add extra space to root partition

set -e

set src= before pulling in host config

no sh -x

fix msdos mbr partition size calculation

don't dup -F

add -F to fdisk

mass -N db to makefs too

specify a database for mtree.

dont call mtree directly

really calling disklabel with the arguments of fdisk does not work.

- remove dup trap
- compare size to 0 for autosizing
- remove .gz file before compressing.

add compression support.

add tool hooks.

fix usage again -- -D specifies destdir not releasedir
include conf/${h}.conf not conf/rpi.conf
fix ./mkimage: line 110: "7" - 1 : syntax error: operand expected (error token is ""7" - 1 ")

fix usage

fix options

switch to makefs

Add the tests sets by default. Tweak the usage output.

More usage updates

Update usage output a little

- Add support for xsets
- Factor out common evbarm code

tidy up some more by factoring out the sizing code.

Make autosizing really work:
- Use newfs to compute the actual filesystem required size.
- Fix the computation of set sizes to account for blocks and fragments.
This results into a 95% full filesystem for my test (Total 566MB, 24MB free)
from which 8MB is my requested overhead. Not perfect, but good enough.

first step at autosizing, still does not work.

expand set name to path when checking to see if the tgz exists

move 'trap' after command args are parsed

honor ${TMPDIR}

complain on unknown options

move configuration to new place.

Adjust mkimage to handle RPI. From skrll@

branches: 1.3.2; 1.3.4;
consolidate all of the volatile /var.* directories which are mounted as tmpfs
into a single one for /var - same as /root, /etc

While the resulting image from mkimage can always be mounted with vnd,
and then modified, it is much easier to let this late, user-oriented
customisation be done automatically. These mods let mkimage change
information based on files in a "custom" directory, which can be set
in mkimage using -c customdir.

After all other configuration, the files under custom/ will be copied
into the main image. The files under custom/ are taken into account
when performing autosizing.

Also, generalise the sets directory to point to the current machine
architecture, although this can be overwritten from the command line
by anyone making cross-images.

An example to make a user image with an embedded read-only 100 MB
iSCSI target:

% ./mkimage -c custom
=== making a new 272 MB image in usermode.img ===
272+0 records in
272+0 records out
285212672 bytes transferred in 7.616 secs (37449142 bytes/sec)
=== mounting image via vnd vnd0 ===
/dev/rvnd0a: 272.0MB (557056 sectors) block size 8192, fragment size 1024
using 6 cylinder groups of 45.34MB, 5803 blks, 11264 inodes.
super-block backups (for fsck_ffs -b #) at:
32, 92880, 185728, 278576, 371424, 464272,
=== installing sets ===
base
etc
modules
=== performing customisations ===
=== making extra directories ===
=== customising /var/tmp ===
=== user customisations from files in custom ===
Filesystem 1K-blocks Used Avail %Cap Mounted on
/dev/vnd0a 269967 263958 -7489 102% /mnt
% ls -l -R custom
total 8
drwxr-xr-x 3 agc agc 512 Jan 18 03:58 etc
drwxr-xr-x 2 agc agc 512 Jan 18 04:00 iscsi

custom/etc:
total 8
drwxr-xr-x 2 agc agc 512 Jan 18 03:58 iscsi
-rw-r--r-- 1 agc agc 564 Jan 18 03:58 rc.conf

custom/etc/iscsi:
total 6
-r--r--r-- 1 agc agc 799 Jan 18 04:27 targets

custom/iscsi:
total 102484
-rw-r--r-- 1 agc agc 104857600 Jan 18 04:00 iscsi-target0
% sudo vnconfig vnd0 usermode.img
% sudo mount /dev/vnd0a /mnt
% cat /mnt/etc/iscsi/targets
# $NetBSD: targets,v 1.2 2006/02/16 19:30:57 agc Exp $
#
# Structure of this file:
#
# + an extent is a straight (offset, length) pair of a file or device
# it's the lowest common storage denominator
# at least one is needed
# + a device is made up of one or more extents or other devices
# devices can be added in a hierachical manner, to enhance resilience
# + in this example, no device definitions are necessary, as the target
# will just use a simple extent for persistent storage
# + a target is made up of 1 or more devices
# The code does not support RAID1 recovery at present

# Simple file showing 1 extent, mapped straight into 1 target

# extent file or device start length
extent0 /iscsi/iscsi-target0 0 100MB

# target flags storage netmask
target0 ro extent0 0.0.0.0/0
% ls -l /mnt/iscsi/iscsi-target0
-rw-r--r-- 1 agc agc 104857600 Jan 18 04:00 /mnt/iscsi/iscsi-target0
% diff custom/etc/iscsi/targets /mnt/etc/iscsi/targets
%

Add 3 scripts which are useful for making images for usermode, or any
embedded platform in particular.

+ mkimage will make an autosized image in a file (using vnd) from sets
the resulting image can be mounted read-only, with tmpfs used for volatile
files on top of read-only bases

+ mkpkgs will make an autosized image in a file (using vnd) from binary
packages

+ usermode is an easy way to invoke usermode, making sure that bridging
is set up properly, and that the host syscallemu module is loaded. This
script then runs usermode with two file images (as produced by mkimage
and mkpkgs)

Quote argument to -n as required. [ -n ] is not a syntax error but
evaluates to true.

PR/57925: Jan-Benedict Glaw: Honor $MKREPRO_TIMESTAMP, don't embed name and
stamp in gzip.

evbppc: Add initial support for the Nintendo Wii

branches: 1.78.2;
Move DRM-driver firmware from base to its own set, gpufw

This set is only installed on amd64,i386,evbarm.
This set is installed on minimal installs and on install media, in
case someone needs it for basic driver functionality.

Comments:
Switched to a single MK tunable for it - that is probably unneeded.

An upcoming DRM update will include even fatter firmware, and we'd
like to minimize the impact of it.

Fix fallout from mkimage rev 1.76.

For mkimage:

- Update "size" if auto-calculated.
- Use "dd bs=1" instead of non-portable "head -c".
- Some style nits.

For MD make_label() functions:

- Stop using "newsize" as image size in MB, use "size" instead.

embedded/mkimage: Pad image with zeros to multiple of 4 MB plus 1 MB.

Otherwise, there may not be enough space after the ffs partition for a
gpt, leading to very confusing results.

Create an "octeon.img.gz" bootable image for evbmips64-eb release builds.

Add support for hybrid MBR/GPT images.

Add GPT support to mkimage.

Separate devicetree .dts -> .dtb building from kernel builds. They are now
part of a separate set, "dtb.tgz", and only built when MKDTB=yes. This
defaults to yes for earmv[67]* and aarch64, and no everywhere else.

Derive the FAT type from the MBR partition type and pass it on to makefs
when crating a msdos file system.

Split out /rescue to its own set and adapt installers/images to add it.

This is meant to make updates safer: if something goes wrong with updating
base, we still have the old, standalone /rescue to recover from.

branches: 1.69.4; 1.69.6;

Be more precise about exactly what fails when something does.

Relying upon set -e to abort things is sort of OK (it is not
a recommended option to use in general - too many odd special cases),
but only if user can work out from the "build failed" what actually
went wrong.

Tested only on amd64 build (for this, i386 is the same) - if anyone
has problems on builds for other systems, please let me know. However
the changes affect only failure paths, the most likely problem would
be for a build to fail to halt on an error, and I hope I have avoided
that. There should be no difference at all to error-free builds.

Handle use of TMPDIR in cleanup as well.

Honour ${TMPDIR} if set for location of image, rather than always
simply using /tmp

Replace HUMMINGBIRD_A31 with SUNXI kernel on armv7.img and include .dtb
files for SUNXI and TEGRA kernels on the MSDOS partition.

add the tests set.

branches: 1.64.4;
Now that FAT+FFS images include the BSD partition in the MBR partition
table, install the disklabel after setting up MBR partitions. This moves
the disklabel to the BSD partition from the start of the disk, avoiding
a conflict with the ODROID-C1 bootloader.

Improvements and fixes for FAT+FFS images:
- Add a "NETBSD" volume label to the MSDOS file-system.
- Explicitly initialize boot sector.
- Correct an issue where the MSDOS file-system was larger than the
partition table entry.
- Add NetBSD partition to the MBR partition table.

Mark the msdos parition active for boards like the BeagleBone Black

branches: 1.61.2; 1.61.4;
use a larger (64KB) block size, this helps SD card performance

Add a new endianes flag to mkimage and pass it on big endian arm builds.
From Rin Okuyama.

fix mkubootimage

Add some documentation:
Tell what this does, and what it expects from conf files

Properly spell X' name. From X(7):

The X.Org Foundation requests that the following names be used when
referring to this software:

X
X Window System
X Version 11
X Window System, Version 11
X11

initialize resize before pulling in board config

Add support for auto-growing the root partition. Enable it for rpi.img.

resizing of ffsv2 still doesn't work well, so revert previous, and instead
explicitly set fsize=2048/bsize=16384, which gives a pretty decent boost
to resize_ffs performance on its own

Use FFSv2 instead of FFSv1 for the root file-system. This should fix the
"resize_ffs takes forever" issues with evbarm images -- growing a v2
file-system is much faster than v1.

Fix previous so that the new generated rc.conf is picked up and that
any new new mount points (e.g. /proc and /kern) are generated.

While here increate the fs size so that postfix can actually build
/etc/mail/aliases.db

PR/49596: mkimage: incorrect permissions on evbarm images

Use NetBSD.dist in our mtree specification to populate the top level
directories, etc.

For each generated file hand craft an mtree entry.

Remove specialdirs as NetBSD.dist creates /proc and /kern.

Use [dx]sets_p as flag for debug/X sets inclusion. xsets is the list of
sets.

fix image building:
- round up the total size of the disk image
- fix fdisk offset for netbsd ffs images

branches: 1.48.4;
now that fdisk has optional arguments, don't pass -1

provide a -r rootdev option to handle the wd/sd/ld lossage.

these changes produce a bootable system under qemu, but its root is trashed.

Add preliminary support for creating x86 usb sticks with mkimage.
This is untested, and just a checkpoint of the work. Image looks
ok, but I have not tried if it boots.

Fix build release of evbarm which had been failing in mkimage
beagleboard_smp with the elusive message:

"gzip: can't stat: gzip: No such file or directory"

This was due to GZIP=gzip being defined in mkimage; we now use:

GZIP_CMD=${TOOL_GZIP:-gzip} # ${GZIP} is special to gzip(1)

as distrib/sets/sets.subr already does in these files as well:

distrib/utils/embedded/mkimage
distrib/utils/embedded/conf/beagleboard.conf
distrib/utils/embedded/conf/rpi.conf
distrib/utils/embedded/conf/rpi_inst.conf

SSD cards is stupid.

explain -m better

tidy up, fix usage

Use TOOL_GZIP in mkimage, and pass the TOOL_GZIP into it.

- explanations for single letter flags
- flag to add debug sets
- flag to minimize writes to sd's

Double extra to fix image size to fix build.

For ffs, set bytes per inode to 4K. Otherwise we get a density based on
the contents of the image, which aren't very useful after the file-system
has been resized -- for rpi.img, this was 22K per inode!

Add a new installer image for Raspberry PI to evbarm

Includes the following major changes:

1) Add new configmenu item to sysinst to allow creation of users at
post-install time.
2) Add an sshramdisk type to evbarm, which is a ramdisk with sshd
enabled, allowing users to install on a headless PI by sshing to the
device with the account "sysinst" password "netbsd" to begin the
installation. (Note, neither the account, nor the ssh setup are copied to
the finalized installation image)
3) Change mkimage to build a boot-only image, without the root filesystems.

Much of the sshramdisk code could be reused on other ports easily.

don't use [ -f $i ] && cat $i, since we are using -e

add extra space to root partition

set -e

set src= before pulling in host config

no sh -x

fix msdos mbr partition size calculation

don't dup -F

add -F to fdisk

mass -N db to makefs too

specify a database for mtree.

dont call mtree directly

really calling disklabel with the arguments of fdisk does not work.

- remove dup trap
- compare size to 0 for autosizing
- remove .gz file before compressing.

add compression support.

add tool hooks.

fix usage again -- -D specifies destdir not releasedir
include conf/${h}.conf not conf/rpi.conf
fix ./mkimage: line 110: "7" - 1 : syntax error: operand expected (error token is ""7" - 1 ")

fix usage

fix options

switch to makefs

Add the tests sets by default. Tweak the usage output.

More usage updates

Update usage output a little

- Add support for xsets
- Factor out common evbarm code

tidy up some more by factoring out the sizing code.

Make autosizing really work:
- Use newfs to compute the actual filesystem required size.
- Fix the computation of set sizes to account for blocks and fragments.
This results into a 95% full filesystem for my test (Total 566MB, 24MB free)
from which 8MB is my requested overhead. Not perfect, but good enough.

first step at autosizing, still does not work.

expand set name to path when checking to see if the tgz exists

move 'trap' after command args are parsed

honor ${TMPDIR}

complain on unknown options

move configuration to new place.

Adjust mkimage to handle RPI. From skrll@

branches: 1.3.2; 1.3.4;
consolidate all of the volatile /var.* directories which are mounted as tmpfs
into a single one for /var - same as /root, /etc

While the resulting image from mkimage can always be mounted with vnd,
and then modified, it is much easier to let this late, user-oriented
customisation be done automatically. These mods let mkimage change
information based on files in a "custom" directory, which can be set
in mkimage using -c customdir.

After all other configuration, the files under custom/ will be copied
into the main image. The files under custom/ are taken into account
when performing autosizing.

Also, generalise the sets directory to point to the current machine
architecture, although this can be overwritten from the command line
by anyone making cross-images.

An example to make a user image with an embedded read-only 100 MB
iSCSI target:

% ./mkimage -c custom
=== making a new 272 MB image in usermode.img ===
272+0 records in
272+0 records out
285212672 bytes transferred in 7.616 secs (37449142 bytes/sec)
=== mounting image via vnd vnd0 ===
/dev/rvnd0a: 272.0MB (557056 sectors) block size 8192, fragment size 1024
using 6 cylinder groups of 45.34MB, 5803 blks, 11264 inodes.
super-block backups (for fsck_ffs -b #) at:
32, 92880, 185728, 278576, 371424, 464272,
=== installing sets ===
base
etc
modules
=== performing customisations ===
=== making extra directories ===
=== customising /var/tmp ===
=== user customisations from files in custom ===
Filesystem 1K-blocks Used Avail %Cap Mounted on
/dev/vnd0a 269967 263958 -7489 102% /mnt
% ls -l -R custom
total 8
drwxr-xr-x 3 agc agc 512 Jan 18 03:58 etc
drwxr-xr-x 2 agc agc 512 Jan 18 04:00 iscsi

custom/etc:
total 8
drwxr-xr-x 2 agc agc 512 Jan 18 03:58 iscsi
-rw-r--r-- 1 agc agc 564 Jan 18 03:58 rc.conf

custom/etc/iscsi:
total 6
-r--r--r-- 1 agc agc 799 Jan 18 04:27 targets

custom/iscsi:
total 102484
-rw-r--r-- 1 agc agc 104857600 Jan 18 04:00 iscsi-target0
% sudo vnconfig vnd0 usermode.img
% sudo mount /dev/vnd0a /mnt
% cat /mnt/etc/iscsi/targets
# $NetBSD: targets,v 1.2 2006/02/16 19:30:57 agc Exp $
#
# Structure of this file:
#
# + an extent is a straight (offset, length) pair of a file or device
# it's the lowest common storage denominator
# at least one is needed
# + a device is made up of one or more extents or other devices
# devices can be added in a hierachical manner, to enhance resilience
# + in this example, no device definitions are necessary, as the target
# will just use a simple extent for persistent storage
# + a target is made up of 1 or more devices
# The code does not support RAID1 recovery at present

# Simple file showing 1 extent, mapped straight into 1 target

# extent file or device start length
extent0 /iscsi/iscsi-target0 0 100MB

# target flags storage netmask
target0 ro extent0 0.0.0.0/0
% ls -l /mnt/iscsi/iscsi-target0
-rw-r--r-- 1 agc agc 104857600 Jan 18 04:00 /mnt/iscsi/iscsi-target0
% diff custom/etc/iscsi/targets /mnt/etc/iscsi/targets
%

Add 3 scripts which are useful for making images for usermode, or any
embedded platform in particular.

+ mkimage will make an autosized image in a file (using vnd) from sets
the resulting image can be mounted read-only, with tmpfs used for volatile
files on top of read-only bases

+ mkpkgs will make an autosized image in a file (using vnd) from binary
packages

+ usermode is an easy way to invoke usermode, making sure that bridging
is set up properly, and that the host syscallemu module is loaded. This
script then runs usermode with two file images (as produced by mkimage
and mkpkgs)

evbppc: Add initial support for the Nintendo Wii

Move DRM-driver firmware from base to its own set, gpufw

This set is only installed on amd64,i386,evbarm.
This set is installed on minimal installs and on install media, in
case someone needs it for basic driver functionality.

Comments:
Switched to a single MK tunable for it - that is probably unneeded.

An upcoming DRM update will include even fatter firmware, and we'd
like to minimize the impact of it.

Fix fallout from mkimage rev 1.76.

For mkimage:

- Update "size" if auto-calculated.
- Use "dd bs=1" instead of non-portable "head -c".
- Some style nits.

For MD make_label() functions:

- Stop using "newsize" as image size in MB, use "size" instead.

embedded/mkimage: Pad image with zeros to multiple of 4 MB plus 1 MB.

Otherwise, there may not be enough space after the ffs partition for a
gpt, leading to very confusing results.

Create an "octeon.img.gz" bootable image for evbmips64-eb release builds.

Add support for hybrid MBR/GPT images.

Add GPT support to mkimage.

Separate devicetree .dts -> .dtb building from kernel builds. They are now
part of a separate set, "dtb.tgz", and only built when MKDTB=yes. This
defaults to yes for earmv[67]* and aarch64, and no everywhere else.

Derive the FAT type from the MBR partition type and pass it on to makefs
when crating a msdos file system.

Split out /rescue to its own set and adapt installers/images to add it.

This is meant to make updates safer: if something goes wrong with updating
base, we still have the old, standalone /rescue to recover from.

branches: 1.69.4; 1.69.6;

Be more precise about exactly what fails when something does.

Relying upon set -e to abort things is sort of OK (it is not
a recommended option to use in general - too many odd special cases),
but only if user can work out from the "build failed" what actually
went wrong.

Tested only on amd64 build (for this, i386 is the same) - if anyone
has problems on builds for other systems, please let me know. However
the changes affect only failure paths, the most likely problem would
be for a build to fail to halt on an error, and I hope I have avoided
that. There should be no difference at all to error-free builds.

Handle use of TMPDIR in cleanup as well.

Honour ${TMPDIR} if set for location of image, rather than always
simply using /tmp

Replace HUMMINGBIRD_A31 with SUNXI kernel on armv7.img and include .dtb
files for SUNXI and TEGRA kernels on the MSDOS partition.

add the tests set.

branches: 1.64.4;
Now that FAT+FFS images include the BSD partition in the MBR partition
table, install the disklabel after setting up MBR partitions. This moves
the disklabel to the BSD partition from the start of the disk, avoiding
a conflict with the ODROID-C1 bootloader.

Improvements and fixes for FAT+FFS images:
- Add a "NETBSD" volume label to the MSDOS file-system.
- Explicitly initialize boot sector.
- Correct an issue where the MSDOS file-system was larger than the
partition table entry.
- Add NetBSD partition to the MBR partition table.

Mark the msdos parition active for boards like the BeagleBone Black

branches: 1.61.2; 1.61.4;
use a larger (64KB) block size, this helps SD card performance

Add a new endianes flag to mkimage and pass it on big endian arm builds.
From Rin Okuyama.

fix mkubootimage

Add some documentation:
Tell what this does, and what it expects from conf files

Properly spell X' name. From X(7):

The X.Org Foundation requests that the following names be used when
referring to this software:

X
X Window System
X Version 11
X Window System, Version 11
X11

initialize resize before pulling in board config

Add support for auto-growing the root partition. Enable it for rpi.img.

resizing of ffsv2 still doesn't work well, so revert previous, and instead
explicitly set fsize=2048/bsize=16384, which gives a pretty decent boost
to resize_ffs performance on its own

Use FFSv2 instead of FFSv1 for the root file-system. This should fix the
"resize_ffs takes forever" issues with evbarm images -- growing a v2
file-system is much faster than v1.

Fix previous so that the new generated rc.conf is picked up and that
any new new mount points (e.g. /proc and /kern) are generated.

While here increate the fs size so that postfix can actually build
/etc/mail/aliases.db

PR/49596: mkimage: incorrect permissions on evbarm images

Use NetBSD.dist in our mtree specification to populate the top level
directories, etc.

For each generated file hand craft an mtree entry.

Remove specialdirs as NetBSD.dist creates /proc and /kern.

Use [dx]sets_p as flag for debug/X sets inclusion. xsets is the list of
sets.

fix image building:
- round up the total size of the disk image
- fix fdisk offset for netbsd ffs images

branches: 1.48.4;
now that fdisk has optional arguments, don't pass -1

provide a -r rootdev option to handle the wd/sd/ld lossage.

these changes produce a bootable system under qemu, but its root is trashed.

Add preliminary support for creating x86 usb sticks with mkimage.
This is untested, and just a checkpoint of the work. Image looks
ok, but I have not tried if it boots.

Fix build release of evbarm which had been failing in mkimage
beagleboard_smp with the elusive message:

"gzip: can't stat: gzip: No such file or directory"

This was due to GZIP=gzip being defined in mkimage; we now use:

GZIP_CMD=${TOOL_GZIP:-gzip} # ${GZIP} is special to gzip(1)

as distrib/sets/sets.subr already does in these files as well:

distrib/utils/embedded/mkimage
distrib/utils/embedded/conf/beagleboard.conf
distrib/utils/embedded/conf/rpi.conf
distrib/utils/embedded/conf/rpi_inst.conf

SSD cards is stupid.

explain -m better

tidy up, fix usage

Use TOOL_GZIP in mkimage, and pass the TOOL_GZIP into it.

- explanations for single letter flags
- flag to add debug sets
- flag to minimize writes to sd's

Double extra to fix image size to fix build.

For ffs, set bytes per inode to 4K. Otherwise we get a density based on
the contents of the image, which aren't very useful after the file-system
has been resized -- for rpi.img, this was 22K per inode!

Add a new installer image for Raspberry PI to evbarm

Includes the following major changes:

1) Add new configmenu item to sysinst to allow creation of users at
post-install time.
2) Add an sshramdisk type to evbarm, which is a ramdisk with sshd
enabled, allowing users to install on a headless PI by sshing to the
device with the account "sysinst" password "netbsd" to begin the
installation. (Note, neither the account, nor the ssh setup are copied to
the finalized installation image)
3) Change mkimage to build a boot-only image, without the root filesystems.

Much of the sshramdisk code could be reused on other ports easily.

don't use [ -f $i ] && cat $i, since we are using -e

add extra space to root partition

set -e

set src= before pulling in host config

no sh -x

fix msdos mbr partition size calculation

don't dup -F

add -F to fdisk

mass -N db to makefs too

specify a database for mtree.

dont call mtree directly

really calling disklabel with the arguments of fdisk does not work.

- remove dup trap
- compare size to 0 for autosizing
- remove .gz file before compressing.

add compression support.

add tool hooks.

fix usage again -- -D specifies destdir not releasedir
include conf/${h}.conf not conf/rpi.conf
fix ./mkimage: line 110: "7" - 1 : syntax error: operand expected (error token is ""7" - 1 ")

fix usage

fix options

switch to makefs

Add the tests sets by default. Tweak the usage output.

More usage updates

Update usage output a little

- Add support for xsets
- Factor out common evbarm code

tidy up some more by factoring out the sizing code.

Make autosizing really work:
- Use newfs to compute the actual filesystem required size.
- Fix the computation of set sizes to account for blocks and fragments.
This results into a 95% full filesystem for my test (Total 566MB, 24MB free)
from which 8MB is my requested overhead. Not perfect, but good enough.

first step at autosizing, still does not work.

expand set name to path when checking to see if the tgz exists

move 'trap' after command args are parsed

honor ${TMPDIR}

complain on unknown options

move configuration to new place.

Adjust mkimage to handle RPI. From skrll@

branches: 1.3.2; 1.3.4;
consolidate all of the volatile /var.* directories which are mounted as tmpfs
into a single one for /var - same as /root, /etc

While the resulting image from mkimage can always be mounted with vnd,
and then modified, it is much easier to let this late, user-oriented
customisation be done automatically. These mods let mkimage change
information based on files in a "custom" directory, which can be set
in mkimage using -c customdir.

After all other configuration, the files under custom/ will be copied
into the main image. The files under custom/ are taken into account
when performing autosizing.

Also, generalise the sets directory to point to the current machine
architecture, although this can be overwritten from the command line
by anyone making cross-images.

An example to make a user image with an embedded read-only 100 MB
iSCSI target:

% ./mkimage -c custom
=== making a new 272 MB image in usermode.img ===
272+0 records in
272+0 records out
285212672 bytes transferred in 7.616 secs (37449142 bytes/sec)
=== mounting image via vnd vnd0 ===
/dev/rvnd0a: 272.0MB (557056 sectors) block size 8192, fragment size 1024
using 6 cylinder groups of 45.34MB, 5803 blks, 11264 inodes.
super-block backups (for fsck_ffs -b #) at:
32, 92880, 185728, 278576, 371424, 464272,
=== installing sets ===
base
etc
modules
=== performing customisations ===
=== making extra directories ===
=== customising /var/tmp ===
=== user customisations from files in custom ===
Filesystem 1K-blocks Used Avail %Cap Mounted on
/dev/vnd0a 269967 263958 -7489 102% /mnt
% ls -l -R custom
total 8
drwxr-xr-x 3 agc agc 512 Jan 18 03:58 etc
drwxr-xr-x 2 agc agc 512 Jan 18 04:00 iscsi

custom/etc:
total 8
drwxr-xr-x 2 agc agc 512 Jan 18 03:58 iscsi
-rw-r--r-- 1 agc agc 564 Jan 18 03:58 rc.conf

custom/etc/iscsi:
total 6
-r--r--r-- 1 agc agc 799 Jan 18 04:27 targets

custom/iscsi:
total 102484
-rw-r--r-- 1 agc agc 104857600 Jan 18 04:00 iscsi-target0
% sudo vnconfig vnd0 usermode.img
% sudo mount /dev/vnd0a /mnt
% cat /mnt/etc/iscsi/targets
# $NetBSD: targets,v 1.2 2006/02/16 19:30:57 agc Exp $
#
# Structure of this file:
#
# + an extent is a straight (offset, length) pair of a file or device
# it's the lowest common storage denominator
# at least one is needed
# + a device is made up of one or more extents or other devices
# devices can be added in a hierachical manner, to enhance resilience
# + in this example, no device definitions are necessary, as the target
# will just use a simple extent for persistent storage
# + a target is made up of 1 or more devices
# The code does not support RAID1 recovery at present

# Simple file showing 1 extent, mapped straight into 1 target

# extent file or device start length
extent0 /iscsi/iscsi-target0 0 100MB

# target flags storage netmask
target0 ro extent0 0.0.0.0/0
% ls -l /mnt/iscsi/iscsi-target0
-rw-r--r-- 1 agc agc 104857600 Jan 18 04:00 /mnt/iscsi/iscsi-target0
% diff custom/etc/iscsi/targets /mnt/etc/iscsi/targets
%

Add 3 scripts which are useful for making images for usermode, or any
embedded platform in particular.

+ mkimage will make an autosized image in a file (using vnd) from sets
the resulting image can be mounted read-only, with tmpfs used for volatile
files on top of read-only bases

+ mkpkgs will make an autosized image in a file (using vnd) from binary
packages

+ usermode is an easy way to invoke usermode, making sure that bridging
is set up properly, and that the host syscallemu module is loaded. This
script then runs usermode with two file images (as produced by mkimage
and mkpkgs)

Move DRM-driver firmware from base to its own set, gpufw

This set is only installed on amd64,i386,evbarm.
This set is installed on minimal installs and on install media, in
case someone needs it for basic driver functionality.

Comments:
Switched to a single MK tunable for it - that is probably unneeded.

An upcoming DRM update will include even fatter firmware, and we'd
like to minimize the impact of it.

Fix fallout from mkimage rev 1.76.

For mkimage:

- Update "size" if auto-calculated.
- Use "dd bs=1" instead of non-portable "head -c".
- Some style nits.

For MD make_label() functions:

- Stop using "newsize" as image size in MB, use "size" instead.

embedded/mkimage: Pad image with zeros to multiple of 4 MB plus 1 MB.

Otherwise, there may not be enough space after the ffs partition for a
gpt, leading to very confusing results.

Create an "octeon.img.gz" bootable image for evbmips64-eb release builds.

Add support for hybrid MBR/GPT images.

Add GPT support to mkimage.

Separate devicetree .dts -> .dtb building from kernel builds. They are now
part of a separate set, "dtb.tgz", and only built when MKDTB=yes. This
defaults to yes for earmv[67]* and aarch64, and no everywhere else.

Derive the FAT type from the MBR partition type and pass it on to makefs
when crating a msdos file system.

Split out /rescue to its own set and adapt installers/images to add it.

This is meant to make updates safer: if something goes wrong with updating
base, we still have the old, standalone /rescue to recover from.

branches: 1.69.4; 1.69.6;

Be more precise about exactly what fails when something does.

Relying upon set -e to abort things is sort of OK (it is not
a recommended option to use in general - too many odd special cases),
but only if user can work out from the "build failed" what actually
went wrong.

Tested only on amd64 build (for this, i386 is the same) - if anyone
has problems on builds for other systems, please let me know. However
the changes affect only failure paths, the most likely problem would
be for a build to fail to halt on an error, and I hope I have avoided
that. There should be no difference at all to error-free builds.

Handle use of TMPDIR in cleanup as well.

Honour ${TMPDIR} if set for location of image, rather than always
simply using /tmp

Replace HUMMINGBIRD_A31 with SUNXI kernel on armv7.img and include .dtb
files for SUNXI and TEGRA kernels on the MSDOS partition.

add the tests set.

branches: 1.64.4;
Now that FAT+FFS images include the BSD partition in the MBR partition
table, install the disklabel after setting up MBR partitions. This moves
the disklabel to the BSD partition from the start of the disk, avoiding
a conflict with the ODROID-C1 bootloader.

Improvements and fixes for FAT+FFS images:
- Add a "NETBSD" volume label to the MSDOS file-system.
- Explicitly initialize boot sector.
- Correct an issue where the MSDOS file-system was larger than the
partition table entry.
- Add NetBSD partition to the MBR partition table.

Mark the msdos parition active for boards like the BeagleBone Black

branches: 1.61.2; 1.61.4;
use a larger (64KB) block size, this helps SD card performance

Add a new endianes flag to mkimage and pass it on big endian arm builds.
From Rin Okuyama.

fix mkubootimage

Add some documentation:
Tell what this does, and what it expects from conf files

Properly spell X' name. From X(7):

The X.Org Foundation requests that the following names be used when
referring to this software:

X
X Window System
X Version 11
X Window System, Version 11
X11

initialize resize before pulling in board config

Add support for auto-growing the root partition. Enable it for rpi.img.

resizing of ffsv2 still doesn't work well, so revert previous, and instead
explicitly set fsize=2048/bsize=16384, which gives a pretty decent boost
to resize_ffs performance on its own

Use FFSv2 instead of FFSv1 for the root file-system. This should fix the
"resize_ffs takes forever" issues with evbarm images -- growing a v2
file-system is much faster than v1.

Fix previous so that the new generated rc.conf is picked up and that
any new new mount points (e.g. /proc and /kern) are generated.

While here increate the fs size so that postfix can actually build
/etc/mail/aliases.db

PR/49596: mkimage: incorrect permissions on evbarm images

Use NetBSD.dist in our mtree specification to populate the top level
directories, etc.

For each generated file hand craft an mtree entry.

Remove specialdirs as NetBSD.dist creates /proc and /kern.

Use [dx]sets_p as flag for debug/X sets inclusion. xsets is the list of
sets.

fix image building:
- round up the total size of the disk image
- fix fdisk offset for netbsd ffs images

branches: 1.48.4;
now that fdisk has optional arguments, don't pass -1

provide a -r rootdev option to handle the wd/sd/ld lossage.

these changes produce a bootable system under qemu, but its root is trashed.

Add preliminary support for creating x86 usb sticks with mkimage.
This is untested, and just a checkpoint of the work. Image looks
ok, but I have not tried if it boots.

Fix build release of evbarm which had been failing in mkimage
beagleboard_smp with the elusive message:

"gzip: can't stat: gzip: No such file or directory"

This was due to GZIP=gzip being defined in mkimage; we now use:

GZIP_CMD=${TOOL_GZIP:-gzip} # ${GZIP} is special to gzip(1)

as distrib/sets/sets.subr already does in these files as well:

distrib/utils/embedded/mkimage
distrib/utils/embedded/conf/beagleboard.conf
distrib/utils/embedded/conf/rpi.conf
distrib/utils/embedded/conf/rpi_inst.conf

SSD cards is stupid.

explain -m better

tidy up, fix usage

Use TOOL_GZIP in mkimage, and pass the TOOL_GZIP into it.

- explanations for single letter flags
- flag to add debug sets
- flag to minimize writes to sd's

Double extra to fix image size to fix build.

For ffs, set bytes per inode to 4K. Otherwise we get a density based on
the contents of the image, which aren't very useful after the file-system
has been resized -- for rpi.img, this was 22K per inode!

Add a new installer image for Raspberry PI to evbarm

Includes the following major changes:

1) Add new configmenu item to sysinst to allow creation of users at
post-install time.
2) Add an sshramdisk type to evbarm, which is a ramdisk with sshd
enabled, allowing users to install on a headless PI by sshing to the
device with the account "sysinst" password "netbsd" to begin the
installation. (Note, neither the account, nor the ssh setup are copied to
the finalized installation image)
3) Change mkimage to build a boot-only image, without the root filesystems.

Much of the sshramdisk code could be reused on other ports easily.

don't use [ -f $i ] && cat $i, since we are using -e

add extra space to root partition

set -e

set src= before pulling in host config

no sh -x

fix msdos mbr partition size calculation

don't dup -F

add -F to fdisk

mass -N db to makefs too

specify a database for mtree.

dont call mtree directly

really calling disklabel with the arguments of fdisk does not work.

- remove dup trap
- compare size to 0 for autosizing
- remove .gz file before compressing.

add compression support.

add tool hooks.

fix usage again -- -D specifies destdir not releasedir
include conf/${h}.conf not conf/rpi.conf
fix ./mkimage: line 110: "7" - 1 : syntax error: operand expected (error token is ""7" - 1 ")

fix usage

fix options

switch to makefs

Add the tests sets by default. Tweak the usage output.

More usage updates

Update usage output a little

- Add support for xsets
- Factor out common evbarm code

tidy up some more by factoring out the sizing code.

Make autosizing really work:
- Use newfs to compute the actual filesystem required size.
- Fix the computation of set sizes to account for blocks and fragments.
This results into a 95% full filesystem for my test (Total 566MB, 24MB free)
from which 8MB is my requested overhead. Not perfect, but good enough.

first step at autosizing, still does not work.

expand set name to path when checking to see if the tgz exists

move 'trap' after command args are parsed

honor ${TMPDIR}

complain on unknown options

move configuration to new place.

Adjust mkimage to handle RPI. From skrll@

branches: 1.3.2; 1.3.4;
consolidate all of the volatile /var.* directories which are mounted as tmpfs
into a single one for /var - same as /root, /etc

While the resulting image from mkimage can always be mounted with vnd,
and then modified, it is much easier to let this late, user-oriented
customisation be done automatically. These mods let mkimage change
information based on files in a "custom" directory, which can be set
in mkimage using -c customdir.

After all other configuration, the files under custom/ will be copied
into the main image. The files under custom/ are taken into account
when performing autosizing.

Also, generalise the sets directory to point to the current machine
architecture, although this can be overwritten from the command line
by anyone making cross-images.

An example to make a user image with an embedded read-only 100 MB
iSCSI target:

% ./mkimage -c custom
=== making a new 272 MB image in usermode.img ===
272+0 records in
272+0 records out
285212672 bytes transferred in 7.616 secs (37449142 bytes/sec)
=== mounting image via vnd vnd0 ===
/dev/rvnd0a: 272.0MB (557056 sectors) block size 8192, fragment size 1024
using 6 cylinder groups of 45.34MB, 5803 blks, 11264 inodes.
super-block backups (for fsck_ffs -b #) at:
32, 92880, 185728, 278576, 371424, 464272,
=== installing sets ===
base
etc
modules
=== performing customisations ===
=== making extra directories ===
=== customising /var/tmp ===
=== user customisations from files in custom ===
Filesystem 1K-blocks Used Avail %Cap Mounted on
/dev/vnd0a 269967 263958 -7489 102% /mnt
% ls -l -R custom
total 8
drwxr-xr-x 3 agc agc 512 Jan 18 03:58 etc
drwxr-xr-x 2 agc agc 512 Jan 18 04:00 iscsi

custom/etc:
total 8
drwxr-xr-x 2 agc agc 512 Jan 18 03:58 iscsi
-rw-r--r-- 1 agc agc 564 Jan 18 03:58 rc.conf

custom/etc/iscsi:
total 6
-r--r--r-- 1 agc agc 799 Jan 18 04:27 targets

custom/iscsi:
total 102484
-rw-r--r-- 1 agc agc 104857600 Jan 18 04:00 iscsi-target0
% sudo vnconfig vnd0 usermode.img
% sudo mount /dev/vnd0a /mnt
% cat /mnt/etc/iscsi/targets
# $NetBSD: targets,v 1.2 2006/02/16 19:30:57 agc Exp $
#
# Structure of this file:
#
# + an extent is a straight (offset, length) pair of a file or device
# it's the lowest common storage denominator
# at least one is needed
# + a device is made up of one or more extents or other devices
# devices can be added in a hierachical manner, to enhance resilience
# + in this example, no device definitions are necessary, as the target
# will just use a simple extent for persistent storage
# + a target is made up of 1 or more devices
# The code does not support RAID1 recovery at present

# Simple file showing 1 extent, mapped straight into 1 target

# extent file or device start length
extent0 /iscsi/iscsi-target0 0 100MB

# target flags storage netmask
target0 ro extent0 0.0.0.0/0
% ls -l /mnt/iscsi/iscsi-target0
-rw-r--r-- 1 agc agc 104857600 Jan 18 04:00 /mnt/iscsi/iscsi-target0
% diff custom/etc/iscsi/targets /mnt/etc/iscsi/targets
%

Add 3 scripts which are useful for making images for usermode, or any
embedded platform in particular.

+ mkimage will make an autosized image in a file (using vnd) from sets
the resulting image can be mounted read-only, with tmpfs used for volatile
files on top of read-only bases

+ mkpkgs will make an autosized image in a file (using vnd) from binary
packages

+ usermode is an easy way to invoke usermode, making sure that bridging
is set up properly, and that the host syscallemu module is loaded. This
script then runs usermode with two file images (as produced by mkimage
and mkpkgs)

Fix fallout from mkimage rev 1.76.

For mkimage:

- Update "size" if auto-calculated.
- Use "dd bs=1" instead of non-portable "head -c".
- Some style nits.

For MD make_label() functions:

- Stop using "newsize" as image size in MB, use "size" instead.

embedded/mkimage: Pad image with zeros to multiple of 4 MB plus 1 MB.

Otherwise, there may not be enough space after the ffs partition for a
gpt, leading to very confusing results.

Create an "octeon.img.gz" bootable image for evbmips64-eb release builds.

Add support for hybrid MBR/GPT images.

Add GPT support to mkimage.

Separate devicetree .dts -> .dtb building from kernel builds. They are now
part of a separate set, "dtb.tgz", and only built when MKDTB=yes. This
defaults to yes for earmv[67]* and aarch64, and no everywhere else.

Derive the FAT type from the MBR partition type and pass it on to makefs
when crating a msdos file system.

Split out /rescue to its own set and adapt installers/images to add it.

This is meant to make updates safer: if something goes wrong with updating
base, we still have the old, standalone /rescue to recover from.

branches: 1.69.4; 1.69.6;

Be more precise about exactly what fails when something does.

Relying upon set -e to abort things is sort of OK (it is not
a recommended option to use in general - too many odd special cases),
but only if user can work out from the "build failed" what actually
went wrong.

Tested only on amd64 build (for this, i386 is the same) - if anyone
has problems on builds for other systems, please let me know. However
the changes affect only failure paths, the most likely problem would
be for a build to fail to halt on an error, and I hope I have avoided
that. There should be no difference at all to error-free builds.

Handle use of TMPDIR in cleanup as well.

Honour ${TMPDIR} if set for location of image, rather than always
simply using /tmp

Replace HUMMINGBIRD_A31 with SUNXI kernel on armv7.img and include .dtb
files for SUNXI and TEGRA kernels on the MSDOS partition.

add the tests set.

branches: 1.64.4;
Now that FAT+FFS images include the BSD partition in the MBR partition
table, install the disklabel after setting up MBR partitions. This moves
the disklabel to the BSD partition from the start of the disk, avoiding
a conflict with the ODROID-C1 bootloader.

Improvements and fixes for FAT+FFS images:
- Add a "NETBSD" volume label to the MSDOS file-system.
- Explicitly initialize boot sector.
- Correct an issue where the MSDOS file-system was larger than the
partition table entry.
- Add NetBSD partition to the MBR partition table.

Mark the msdos parition active for boards like the BeagleBone Black

branches: 1.61.2; 1.61.4;
use a larger (64KB) block size, this helps SD card performance

Add a new endianes flag to mkimage and pass it on big endian arm builds.
From Rin Okuyama.

fix mkubootimage

Add some documentation:
Tell what this does, and what it expects from conf files

Properly spell X' name. From X(7):

The X.Org Foundation requests that the following names be used when
referring to this software:

X
X Window System
X Version 11
X Window System, Version 11
X11

initialize resize before pulling in board config

Add support for auto-growing the root partition. Enable it for rpi.img.

resizing of ffsv2 still doesn't work well, so revert previous, and instead
explicitly set fsize=2048/bsize=16384, which gives a pretty decent boost
to resize_ffs performance on its own

Use FFSv2 instead of FFSv1 for the root file-system. This should fix the
"resize_ffs takes forever" issues with evbarm images -- growing a v2
file-system is much faster than v1.

Fix previous so that the new generated rc.conf is picked up and that
any new new mount points (e.g. /proc and /kern) are generated.

While here increate the fs size so that postfix can actually build
/etc/mail/aliases.db

PR/49596: mkimage: incorrect permissions on evbarm images

Use NetBSD.dist in our mtree specification to populate the top level
directories, etc.

For each generated file hand craft an mtree entry.

Remove specialdirs as NetBSD.dist creates /proc and /kern.

Use [dx]sets_p as flag for debug/X sets inclusion. xsets is the list of
sets.

fix image building:
- round up the total size of the disk image
- fix fdisk offset for netbsd ffs images

branches: 1.48.4;
now that fdisk has optional arguments, don't pass -1

provide a -r rootdev option to handle the wd/sd/ld lossage.

these changes produce a bootable system under qemu, but its root is trashed.

Add preliminary support for creating x86 usb sticks with mkimage.
This is untested, and just a checkpoint of the work. Image looks
ok, but I have not tried if it boots.

Fix build release of evbarm which had been failing in mkimage
beagleboard_smp with the elusive message:

"gzip: can't stat: gzip: No such file or directory"

This was due to GZIP=gzip being defined in mkimage; we now use:

GZIP_CMD=${TOOL_GZIP:-gzip} # ${GZIP} is special to gzip(1)

as distrib/sets/sets.subr already does in these files as well:

distrib/utils/embedded/mkimage
distrib/utils/embedded/conf/beagleboard.conf
distrib/utils/embedded/conf/rpi.conf
distrib/utils/embedded/conf/rpi_inst.conf

SSD cards is stupid.

explain -m better

tidy up, fix usage

Use TOOL_GZIP in mkimage, and pass the TOOL_GZIP into it.

- explanations for single letter flags
- flag to add debug sets
- flag to minimize writes to sd's

Double extra to fix image size to fix build.

For ffs, set bytes per inode to 4K. Otherwise we get a density based on
the contents of the image, which aren't very useful after the file-system
has been resized -- for rpi.img, this was 22K per inode!

Add a new installer image for Raspberry PI to evbarm

Includes the following major changes:

1) Add new configmenu item to sysinst to allow creation of users at
post-install time.
2) Add an sshramdisk type to evbarm, which is a ramdisk with sshd
enabled, allowing users to install on a headless PI by sshing to the
device with the account "sysinst" password "netbsd" to begin the
installation. (Note, neither the account, nor the ssh setup are copied to
the finalized installation image)
3) Change mkimage to build a boot-only image, without the root filesystems.

Much of the sshramdisk code could be reused on other ports easily.

don't use [ -f $i ] && cat $i, since we are using -e

add extra space to root partition

set -e

set src= before pulling in host config

no sh -x

fix msdos mbr partition size calculation

don't dup -F

add -F to fdisk

mass -N db to makefs too

specify a database for mtree.

dont call mtree directly

really calling disklabel with the arguments of fdisk does not work.

- remove dup trap
- compare size to 0 for autosizing
- remove .gz file before compressing.

add compression support.

add tool hooks.

fix usage again -- -D specifies destdir not releasedir
include conf/${h}.conf not conf/rpi.conf
fix ./mkimage: line 110: "7" - 1 : syntax error: operand expected (error token is ""7" - 1 ")

fix usage

fix options

switch to makefs

Add the tests sets by default. Tweak the usage output.

More usage updates

Update usage output a little

- Add support for xsets
- Factor out common evbarm code

tidy up some more by factoring out the sizing code.

Make autosizing really work:
- Use newfs to compute the actual filesystem required size.
- Fix the computation of set sizes to account for blocks and fragments.
This results into a 95% full filesystem for my test (Total 566MB, 24MB free)
from which 8MB is my requested overhead. Not perfect, but good enough.

first step at autosizing, still does not work.

expand set name to path when checking to see if the tgz exists

move 'trap' after command args are parsed

honor ${TMPDIR}

complain on unknown options

move configuration to new place.

Adjust mkimage to handle RPI. From skrll@

branches: 1.3.2; 1.3.4;
consolidate all of the volatile /var.* directories which are mounted as tmpfs
into a single one for /var - same as /root, /etc

While the resulting image from mkimage can always be mounted with vnd,
and then modified, it is much easier to let this late, user-oriented
customisation be done automatically. These mods let mkimage change
information based on files in a "custom" directory, which can be set
in mkimage using -c customdir.

After all other configuration, the files under custom/ will be copied
into the main image. The files under custom/ are taken into account
when performing autosizing.

Also, generalise the sets directory to point to the current machine
architecture, although this can be overwritten from the command line
by anyone making cross-images.

An example to make a user image with an embedded read-only 100 MB
iSCSI target:

% ./mkimage -c custom
=== making a new 272 MB image in usermode.img ===
272+0 records in
272+0 records out
285212672 bytes transferred in 7.616 secs (37449142 bytes/sec)
=== mounting image via vnd vnd0 ===
/dev/rvnd0a: 272.0MB (557056 sectors) block size 8192, fragment size 1024
using 6 cylinder groups of 45.34MB, 5803 blks, 11264 inodes.
super-block backups (for fsck_ffs -b #) at:
32, 92880, 185728, 278576, 371424, 464272,
=== installing sets ===
base
etc
modules
=== performing customisations ===
=== making extra directories ===
=== customising /var/tmp ===
=== user customisations from files in custom ===
Filesystem 1K-blocks Used Avail %Cap Mounted on
/dev/vnd0a 269967 263958 -7489 102% /mnt
% ls -l -R custom
total 8
drwxr-xr-x 3 agc agc 512 Jan 18 03:58 etc
drwxr-xr-x 2 agc agc 512 Jan 18 04:00 iscsi

custom/etc:
total 8
drwxr-xr-x 2 agc agc 512 Jan 18 03:58 iscsi
-rw-r--r-- 1 agc agc 564 Jan 18 03:58 rc.conf

custom/etc/iscsi:
total 6
-r--r--r-- 1 agc agc 799 Jan 18 04:27 targets

custom/iscsi:
total 102484
-rw-r--r-- 1 agc agc 104857600 Jan 18 04:00 iscsi-target0
% sudo vnconfig vnd0 usermode.img
% sudo mount /dev/vnd0a /mnt
% cat /mnt/etc/iscsi/targets
# $NetBSD: targets,v 1.2 2006/02/16 19:30:57 agc Exp $
#
# Structure of this file:
#
# + an extent is a straight (offset, length) pair of a file or device
# it's the lowest common storage denominator
# at least one is needed
# + a device is made up of one or more extents or other devices
# devices can be added in a hierachical manner, to enhance resilience
# + in this example, no device definitions are necessary, as the target
# will just use a simple extent for persistent storage
# + a target is made up of 1 or more devices
# The code does not support RAID1 recovery at present

# Simple file showing 1 extent, mapped straight into 1 target

# extent file or device start length
extent0 /iscsi/iscsi-target0 0 100MB

# target flags storage netmask
target0 ro extent0 0.0.0.0/0
% ls -l /mnt/iscsi/iscsi-target0
-rw-r--r-- 1 agc agc 104857600 Jan 18 04:00 /mnt/iscsi/iscsi-target0
% diff custom/etc/iscsi/targets /mnt/etc/iscsi/targets
%

Add 3 scripts which are useful for making images for usermode, or any
embedded platform in particular.

+ mkimage will make an autosized image in a file (using vnd) from sets
the resulting image can be mounted read-only, with tmpfs used for volatile
files on top of read-only bases

+ mkpkgs will make an autosized image in a file (using vnd) from binary
packages

+ usermode is an easy way to invoke usermode, making sure that bridging
is set up properly, and that the host syscallemu module is loaded. This
script then runs usermode with two file images (as produced by mkimage
and mkpkgs)

embedded/mkimage: Pad image with zeros to multiple of 4 MB plus 1 MB.

Otherwise, there may not be enough space after the ffs partition for a
gpt, leading to very confusing results.

Create an "octeon.img.gz" bootable image for evbmips64-eb release builds.

Add support for hybrid MBR/GPT images.

Add GPT support to mkimage.

Separate devicetree .dts -> .dtb building from kernel builds. They are now
part of a separate set, "dtb.tgz", and only built when MKDTB=yes. This
defaults to yes for earmv[67]* and aarch64, and no everywhere else.

Derive the FAT type from the MBR partition type and pass it on to makefs
when crating a msdos file system.

Split out /rescue to its own set and adapt installers/images to add it.

This is meant to make updates safer: if something goes wrong with updating
base, we still have the old, standalone /rescue to recover from.

branches: 1.69.4; 1.69.6;

Be more precise about exactly what fails when something does.

Relying upon set -e to abort things is sort of OK (it is not
a recommended option to use in general - too many odd special cases),
but only if user can work out from the "build failed" what actually
went wrong.

Tested only on amd64 build (for this, i386 is the same) - if anyone
has problems on builds for other systems, please let me know. However
the changes affect only failure paths, the most likely problem would
be for a build to fail to halt on an error, and I hope I have avoided
that. There should be no difference at all to error-free builds.

Handle use of TMPDIR in cleanup as well.

Honour ${TMPDIR} if set for location of image, rather than always
simply using /tmp

Replace HUMMINGBIRD_A31 with SUNXI kernel on armv7.img and include .dtb
files for SUNXI and TEGRA kernels on the MSDOS partition.

add the tests set.

branches: 1.64.4;
Now that FAT+FFS images include the BSD partition in the MBR partition
table, install the disklabel after setting up MBR partitions. This moves
the disklabel to the BSD partition from the start of the disk, avoiding
a conflict with the ODROID-C1 bootloader.

Improvements and fixes for FAT+FFS images:
- Add a "NETBSD" volume label to the MSDOS file-system.
- Explicitly initialize boot sector.
- Correct an issue where the MSDOS file-system was larger than the
partition table entry.
- Add NetBSD partition to the MBR partition table.

Mark the msdos parition active for boards like the BeagleBone Black

branches: 1.61.2; 1.61.4;
use a larger (64KB) block size, this helps SD card performance

Add a new endianes flag to mkimage and pass it on big endian arm builds.
From Rin Okuyama.

fix mkubootimage

Add some documentation:
Tell what this does, and what it expects from conf files

Properly spell X' name. From X(7):

The X.Org Foundation requests that the following names be used when
referring to this software:

X
X Window System
X Version 11
X Window System, Version 11
X11

initialize resize before pulling in board config

Add support for auto-growing the root partition. Enable it for rpi.img.

resizing of ffsv2 still doesn't work well, so revert previous, and instead
explicitly set fsize=2048/bsize=16384, which gives a pretty decent boost
to resize_ffs performance on its own

Use FFSv2 instead of FFSv1 for the root file-system. This should fix the
"resize_ffs takes forever" issues with evbarm images -- growing a v2
file-system is much faster than v1.

Fix previous so that the new generated rc.conf is picked up and that
any new new mount points (e.g. /proc and /kern) are generated.

While here increate the fs size so that postfix can actually build
/etc/mail/aliases.db

PR/49596: mkimage: incorrect permissions on evbarm images

Use NetBSD.dist in our mtree specification to populate the top level
directories, etc.

For each generated file hand craft an mtree entry.

Remove specialdirs as NetBSD.dist creates /proc and /kern.

Use [dx]sets_p as flag for debug/X sets inclusion. xsets is the list of
sets.

fix image building:
- round up the total size of the disk image
- fix fdisk offset for netbsd ffs images

branches: 1.48.4;
now that fdisk has optional arguments, don't pass -1

provide a -r rootdev option to handle the wd/sd/ld lossage.

these changes produce a bootable system under qemu, but its root is trashed.