History log of /openbsd-current/usr.sbin/vmd/vioqcow2.c
Revision (<<< Hide revision tags) (Show revision tags >>>) Date Author Comments
# 1.24 14-Sep-2023 dv

vmd(8)/vioblk: use zero-copy approach & vectored io.

The original version of the virtio block device dynamically allocated
buffers to hold intermediate data when reading or writing to the
underlying disk fd(s). Since vioblk drivers may chain multiple
segments together, this leads to overly complex logic and on
read(2)/write(2) call per data segment.

Additionally, the virtio block logic in vmd didn't handle segments
that weren't block aligned (e.g. 512 bytes). If a guest provided
unaligned segments, garbage will be read or written.

Since virtio descriptors mimic iovec structures, this changes vmd's
device emulation to use that model. (This is how other hypervisors
emulate virtio devices.) This allows for zero-copy semantics using
iovec's, reducing memcpy and multiple read/write syscalls per io
transaction.

Testing by phessler@ and mlarkin@. OK mlarkin@.


# 1.23 28-May-2023 asou

Used number of bytes instead of number of 512 byte sectors.

ok dv@


# 1.22 28-Apr-2023 dv

Remove unneeded header includes in vmd.

No functional change. virtio block/networking emulation do not need
to know about vmm or any kernel types.


# 1.21 27-Apr-2023 dv

vmd(8): introduce multi-process model for virtio devices.

Isolate virtio network and block device emulation in dedicated
processes, forked and exec'd from the vm process. This allows for
tightening pledge promises to just "stdio".

Communication between the vcpu's and these devices now occurs via
imsg channels, which adds the benefit of not always blocking the
vcpu thread while emulating the device.

With this commit, it's possible that vmd is the first open source
hypervisor that *defaults* to a multi-process device emulation
model without requiring any additional configuration from the
operator.

Testing help from phessler@ and Mischa Peters.

ok mlarkin@


Revision tags: OPENBSD_7_2_BASE OPENBSD_7_3_BASE
# 1.20 20-May-2022 dv

use correct logging or error calls

ftruncate sets errno, so use fatal() and not fatalx().

Checking for under-reads of read(2) isn't going to set errno so
use errx().

ok abieber@


# 1.19 13-May-2022 dv

fix typos in comments

No functional change. Pointed out by Martin Vahlensieck.


# 1.18 04-May-2022 dv

vmctl(8)/vmd(8): convert disk sizes from MB to bytes

Continue converting other parts to storing data in bytes instead
of MB. In this case, the logic for disk sizes was being scaled.

This fixes issues reported by Martin Vahlensieck where vmctl could
no longer create disks larger than 7 MiB after previous commits to
change storing memory sizes as bytes.

While this keeps the vm memory limit check in vmctl's size parser,
it skips the limit check for disks. The error messages adjust
accordingly and this removes the double error message logging.

Update comments and function types accordingly.

ok marlkin@


Revision tags: OPENBSD_7_1_BASE
# 1.17 04-Jan-2022 claudio

Another -Wunused-but-set-variable fix.
Based on input from dv@


Revision tags: OPENBSD_7_0_BASE
# 1.16 16-Jun-2021 dv

cleanup vmd(8) includes and header files

Lots of organic growth other the years lead to unnecessary includes
(proc.h everywhere) and odd dependencies between header files. This
cleans things up a bit to help with upcoming cleanup around dhcp
code.

No functional change.

"go for it" mlarkin@


# 1.15 29-Apr-2021 dv

Linting: remove duplicate struct definition (plus whitespace)

"sure" mlarkin@


Revision tags: OPENBSD_6_9_BASE
# 1.14 19-Oct-2020 naddy

Accommodate POSIX basename(3) that takes a non-const parameter and
may in fact modify the string buffer.

truncation check requested and ok florian@


Revision tags: OPENBSD_6_5_BASE OPENBSD_6_6_BASE OPENBSD_6_7_BASE OPENBSD_6_8_BASE
# 1.13 10-Jan-2019 deraadt

an alloca() snuck into the tree. We don't use alloca() in our tree unless
it is entirely unavoidable (for example libc/*/exec.c), because any erroneous
size controlled by an attacker turns into a known-location object placement
in a very dangerous region. So use malloc() instead.


# 1.12 26-Nov-2018 reyk

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

branches: 1.8.2;
Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.23 28-May-2023 asou

Used number of bytes instead of number of 512 byte sectors.

ok dv@


# 1.22 28-Apr-2023 dv

Remove unneeded header includes in vmd.

No functional change. virtio block/networking emulation do not need
to know about vmm or any kernel types.


# 1.21 27-Apr-2023 dv

vmd(8): introduce multi-process model for virtio devices.

Isolate virtio network and block device emulation in dedicated
processes, forked and exec'd from the vm process. This allows for
tightening pledge promises to just "stdio".

Communication between the vcpu's and these devices now occurs via
imsg channels, which adds the benefit of not always blocking the
vcpu thread while emulating the device.

With this commit, it's possible that vmd is the first open source
hypervisor that *defaults* to a multi-process device emulation
model without requiring any additional configuration from the
operator.

Testing help from phessler@ and Mischa Peters.

ok mlarkin@


Revision tags: OPENBSD_7_2_BASE OPENBSD_7_3_BASE
# 1.20 20-May-2022 dv

use correct logging or error calls

ftruncate sets errno, so use fatal() and not fatalx().

Checking for under-reads of read(2) isn't going to set errno so
use errx().

ok abieber@


# 1.19 13-May-2022 dv

fix typos in comments

No functional change. Pointed out by Martin Vahlensieck.


# 1.18 04-May-2022 dv

vmctl(8)/vmd(8): convert disk sizes from MB to bytes

Continue converting other parts to storing data in bytes instead
of MB. In this case, the logic for disk sizes was being scaled.

This fixes issues reported by Martin Vahlensieck where vmctl could
no longer create disks larger than 7 MiB after previous commits to
change storing memory sizes as bytes.

While this keeps the vm memory limit check in vmctl's size parser,
it skips the limit check for disks. The error messages adjust
accordingly and this removes the double error message logging.

Update comments and function types accordingly.

ok marlkin@


Revision tags: OPENBSD_7_1_BASE
# 1.17 04-Jan-2022 claudio

Another -Wunused-but-set-variable fix.
Based on input from dv@


Revision tags: OPENBSD_7_0_BASE
# 1.16 16-Jun-2021 dv

cleanup vmd(8) includes and header files

Lots of organic growth other the years lead to unnecessary includes
(proc.h everywhere) and odd dependencies between header files. This
cleans things up a bit to help with upcoming cleanup around dhcp
code.

No functional change.

"go for it" mlarkin@


# 1.15 29-Apr-2021 dv

Linting: remove duplicate struct definition (plus whitespace)

"sure" mlarkin@


Revision tags: OPENBSD_6_9_BASE
# 1.14 19-Oct-2020 naddy

Accommodate POSIX basename(3) that takes a non-const parameter and
may in fact modify the string buffer.

truncation check requested and ok florian@


Revision tags: OPENBSD_6_5_BASE OPENBSD_6_6_BASE OPENBSD_6_7_BASE OPENBSD_6_8_BASE
# 1.13 10-Jan-2019 deraadt

an alloca() snuck into the tree. We don't use alloca() in our tree unless
it is entirely unavoidable (for example libc/*/exec.c), because any erroneous
size controlled by an attacker turns into a known-location object placement
in a very dangerous region. So use malloc() instead.


# 1.12 26-Nov-2018 reyk

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

branches: 1.8.2;
Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.22 28-Apr-2023 dv

Remove unneeded header includes in vmd.

No functional change. virtio block/networking emulation do not need
to know about vmm or any kernel types.


# 1.21 27-Apr-2023 dv

vmd(8): introduce multi-process model for virtio devices.

Isolate virtio network and block device emulation in dedicated
processes, forked and exec'd from the vm process. This allows for
tightening pledge promises to just "stdio".

Communication between the vcpu's and these devices now occurs via
imsg channels, which adds the benefit of not always blocking the
vcpu thread while emulating the device.

With this commit, it's possible that vmd is the first open source
hypervisor that *defaults* to a multi-process device emulation
model without requiring any additional configuration from the
operator.

Testing help from phessler@ and Mischa Peters.

ok mlarkin@


Revision tags: OPENBSD_7_2_BASE OPENBSD_7_3_BASE
# 1.20 20-May-2022 dv

use correct logging or error calls

ftruncate sets errno, so use fatal() and not fatalx().

Checking for under-reads of read(2) isn't going to set errno so
use errx().

ok abieber@


# 1.19 13-May-2022 dv

fix typos in comments

No functional change. Pointed out by Martin Vahlensieck.


# 1.18 04-May-2022 dv

vmctl(8)/vmd(8): convert disk sizes from MB to bytes

Continue converting other parts to storing data in bytes instead
of MB. In this case, the logic for disk sizes was being scaled.

This fixes issues reported by Martin Vahlensieck where vmctl could
no longer create disks larger than 7 MiB after previous commits to
change storing memory sizes as bytes.

While this keeps the vm memory limit check in vmctl's size parser,
it skips the limit check for disks. The error messages adjust
accordingly and this removes the double error message logging.

Update comments and function types accordingly.

ok marlkin@


Revision tags: OPENBSD_7_1_BASE
# 1.17 04-Jan-2022 claudio

Another -Wunused-but-set-variable fix.
Based on input from dv@


Revision tags: OPENBSD_7_0_BASE
# 1.16 16-Jun-2021 dv

cleanup vmd(8) includes and header files

Lots of organic growth other the years lead to unnecessary includes
(proc.h everywhere) and odd dependencies between header files. This
cleans things up a bit to help with upcoming cleanup around dhcp
code.

No functional change.

"go for it" mlarkin@


# 1.15 29-Apr-2021 dv

Linting: remove duplicate struct definition (plus whitespace)

"sure" mlarkin@


Revision tags: OPENBSD_6_9_BASE
# 1.14 19-Oct-2020 naddy

Accommodate POSIX basename(3) that takes a non-const parameter and
may in fact modify the string buffer.

truncation check requested and ok florian@


Revision tags: OPENBSD_6_5_BASE OPENBSD_6_6_BASE OPENBSD_6_7_BASE OPENBSD_6_8_BASE
# 1.13 10-Jan-2019 deraadt

an alloca() snuck into the tree. We don't use alloca() in our tree unless
it is entirely unavoidable (for example libc/*/exec.c), because any erroneous
size controlled by an attacker turns into a known-location object placement
in a very dangerous region. So use malloc() instead.


# 1.12 26-Nov-2018 reyk

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

branches: 1.8.2;
Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.20 20-May-2022 dv

use correct logging or error calls

ftruncate sets errno, so use fatal() and not fatalx().

Checking for under-reads of read(2) isn't going to set errno so
use errx().

ok abieber@


# 1.19 13-May-2022 dv

fix typos in comments

No functional change. Pointed out by Martin Vahlensieck.


# 1.18 04-May-2022 dv

vmctl(8)/vmd(8): convert disk sizes from MB to bytes

Continue converting other parts to storing data in bytes instead
of MB. In this case, the logic for disk sizes was being scaled.

This fixes issues reported by Martin Vahlensieck where vmctl could
no longer create disks larger than 7 MiB after previous commits to
change storing memory sizes as bytes.

While this keeps the vm memory limit check in vmctl's size parser,
it skips the limit check for disks. The error messages adjust
accordingly and this removes the double error message logging.

Update comments and function types accordingly.

ok marlkin@


Revision tags: OPENBSD_7_1_BASE
# 1.17 04-Jan-2022 claudio

Another -Wunused-but-set-variable fix.
Based on input from dv@


Revision tags: OPENBSD_7_0_BASE
# 1.16 16-Jun-2021 dv

cleanup vmd(8) includes and header files

Lots of organic growth other the years lead to unnecessary includes
(proc.h everywhere) and odd dependencies between header files. This
cleans things up a bit to help with upcoming cleanup around dhcp
code.

No functional change.

"go for it" mlarkin@


# 1.15 29-Apr-2021 dv

Linting: remove duplicate struct definition (plus whitespace)

"sure" mlarkin@


Revision tags: OPENBSD_6_9_BASE
# 1.14 19-Oct-2020 naddy

Accommodate POSIX basename(3) that takes a non-const parameter and
may in fact modify the string buffer.

truncation check requested and ok florian@


Revision tags: OPENBSD_6_5_BASE OPENBSD_6_6_BASE OPENBSD_6_7_BASE OPENBSD_6_8_BASE
# 1.13 10-Jan-2019 deraadt

an alloca() snuck into the tree. We don't use alloca() in our tree unless
it is entirely unavoidable (for example libc/*/exec.c), because any erroneous
size controlled by an attacker turns into a known-location object placement
in a very dangerous region. So use malloc() instead.


# 1.12 26-Nov-2018 reyk

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

branches: 1.8.2;
Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.19 13-May-2022 dv

fix typos in comments

No functional change. Pointed out by Martin Vahlensieck.


# 1.18 04-May-2022 dv

vmctl(8)/vmd(8): convert disk sizes from MB to bytes

Continue converting other parts to storing data in bytes instead
of MB. In this case, the logic for disk sizes was being scaled.

This fixes issues reported by Martin Vahlensieck where vmctl could
no longer create disks larger than 7 MiB after previous commits to
change storing memory sizes as bytes.

While this keeps the vm memory limit check in vmctl's size parser,
it skips the limit check for disks. The error messages adjust
accordingly and this removes the double error message logging.

Update comments and function types accordingly.

ok marlkin@


Revision tags: OPENBSD_7_1_BASE
# 1.17 04-Jan-2022 claudio

Another -Wunused-but-set-variable fix.
Based on input from dv@


Revision tags: OPENBSD_7_0_BASE
# 1.16 16-Jun-2021 dv

cleanup vmd(8) includes and header files

Lots of organic growth other the years lead to unnecessary includes
(proc.h everywhere) and odd dependencies between header files. This
cleans things up a bit to help with upcoming cleanup around dhcp
code.

No functional change.

"go for it" mlarkin@


# 1.15 29-Apr-2021 dv

Linting: remove duplicate struct definition (plus whitespace)

"sure" mlarkin@


Revision tags: OPENBSD_6_9_BASE
# 1.14 19-Oct-2020 naddy

Accommodate POSIX basename(3) that takes a non-const parameter and
may in fact modify the string buffer.

truncation check requested and ok florian@


Revision tags: OPENBSD_6_5_BASE OPENBSD_6_6_BASE OPENBSD_6_7_BASE OPENBSD_6_8_BASE
# 1.13 10-Jan-2019 deraadt

an alloca() snuck into the tree. We don't use alloca() in our tree unless
it is entirely unavoidable (for example libc/*/exec.c), because any erroneous
size controlled by an attacker turns into a known-location object placement
in a very dangerous region. So use malloc() instead.


# 1.12 26-Nov-2018 reyk

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

branches: 1.8.2;
Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.18 04-May-2022 dv

vmctl(8)/vmd(8): convert disk sizes from MB to bytes

Continue converting other parts to storing data in bytes instead
of MB. In this case, the logic for disk sizes was being scaled.

This fixes issues reported by Martin Vahlensieck where vmctl could
no longer create disks larger than 7 MiB after previous commits to
change storing memory sizes as bytes.

While this keeps the vm memory limit check in vmctl's size parser,
it skips the limit check for disks. The error messages adjust
accordingly and this removes the double error message logging.

Update comments and function types accordingly.

ok marlkin@


Revision tags: OPENBSD_7_1_BASE
# 1.17 04-Jan-2022 claudio

Another -Wunused-but-set-variable fix.
Based on input from dv@


Revision tags: OPENBSD_7_0_BASE
# 1.16 16-Jun-2021 dv

cleanup vmd(8) includes and header files

Lots of organic growth other the years lead to unnecessary includes
(proc.h everywhere) and odd dependencies between header files. This
cleans things up a bit to help with upcoming cleanup around dhcp
code.

No functional change.

"go for it" mlarkin@


# 1.15 29-Apr-2021 dv

Linting: remove duplicate struct definition (plus whitespace)

"sure" mlarkin@


Revision tags: OPENBSD_6_9_BASE
# 1.14 19-Oct-2020 naddy

Accommodate POSIX basename(3) that takes a non-const parameter and
may in fact modify the string buffer.

truncation check requested and ok florian@


Revision tags: OPENBSD_6_5_BASE OPENBSD_6_6_BASE OPENBSD_6_7_BASE OPENBSD_6_8_BASE
# 1.13 10-Jan-2019 deraadt

an alloca() snuck into the tree. We don't use alloca() in our tree unless
it is entirely unavoidable (for example libc/*/exec.c), because any erroneous
size controlled by an attacker turns into a known-location object placement
in a very dangerous region. So use malloc() instead.


# 1.12 26-Nov-2018 reyk

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

branches: 1.8.2;
Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.17 04-Jan-2022 claudio

Another -Wunused-but-set-variable fix.
Based on input from dv@


Revision tags: OPENBSD_7_0_BASE
# 1.16 16-Jun-2021 dv

cleanup vmd(8) includes and header files

Lots of organic growth other the years lead to unnecessary includes
(proc.h everywhere) and odd dependencies between header files. This
cleans things up a bit to help with upcoming cleanup around dhcp
code.

No functional change.

"go for it" mlarkin@


# 1.15 29-Apr-2021 dv

Linting: remove duplicate struct definition (plus whitespace)

"sure" mlarkin@


Revision tags: OPENBSD_6_9_BASE
# 1.14 19-Oct-2020 naddy

Accommodate POSIX basename(3) that takes a non-const parameter and
may in fact modify the string buffer.

truncation check requested and ok florian@


Revision tags: OPENBSD_6_5_BASE OPENBSD_6_6_BASE OPENBSD_6_7_BASE OPENBSD_6_8_BASE
# 1.13 10-Jan-2019 deraadt

an alloca() snuck into the tree. We don't use alloca() in our tree unless
it is entirely unavoidable (for example libc/*/exec.c), because any erroneous
size controlled by an attacker turns into a known-location object placement
in a very dangerous region. So use malloc() instead.


# 1.12 26-Nov-2018 reyk

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

branches: 1.8.2;
Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.16 16-Jun-2021 dv

cleanup vmd(8) includes and header files

Lots of organic growth other the years lead to unnecessary includes
(proc.h everywhere) and odd dependencies between header files. This
cleans things up a bit to help with upcoming cleanup around dhcp
code.

No functional change.

"go for it" mlarkin@


# 1.15 29-Apr-2021 dv

Linting: remove duplicate struct definition (plus whitespace)

"sure" mlarkin@


Revision tags: OPENBSD_6_9_BASE
# 1.14 19-Oct-2020 naddy

Accommodate POSIX basename(3) that takes a non-const parameter and
may in fact modify the string buffer.

truncation check requested and ok florian@


Revision tags: OPENBSD_6_5_BASE OPENBSD_6_6_BASE OPENBSD_6_7_BASE OPENBSD_6_8_BASE
# 1.13 10-Jan-2019 deraadt

an alloca() snuck into the tree. We don't use alloca() in our tree unless
it is entirely unavoidable (for example libc/*/exec.c), because any erroneous
size controlled by an attacker turns into a known-location object placement
in a very dangerous region. So use malloc() instead.


# 1.12 26-Nov-2018 reyk

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

branches: 1.8.2;
Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.15 29-Apr-2021 dv

Linting: remove duplicate struct definition (plus whitespace)

"sure" mlarkin@


Revision tags: OPENBSD_6_9_BASE
# 1.14 19-Oct-2020 naddy

Accommodate POSIX basename(3) that takes a non-const parameter and
may in fact modify the string buffer.

truncation check requested and ok florian@


Revision tags: OPENBSD_6_5_BASE OPENBSD_6_6_BASE OPENBSD_6_7_BASE OPENBSD_6_8_BASE
# 1.13 10-Jan-2019 deraadt

an alloca() snuck into the tree. We don't use alloca() in our tree unless
it is entirely unavoidable (for example libc/*/exec.c), because any erroneous
size controlled by an attacker turns into a known-location object placement
in a very dangerous region. So use malloc() instead.


# 1.12 26-Nov-2018 reyk

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

branches: 1.8.2;
Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.14 19-Oct-2020 naddy

Accommodate POSIX basename(3) that takes a non-const parameter and
may in fact modify the string buffer.

truncation check requested and ok florian@


Revision tags: OPENBSD_6_5_BASE OPENBSD_6_6_BASE OPENBSD_6_7_BASE OPENBSD_6_8_BASE
# 1.13 10-Jan-2019 deraadt

an alloca() snuck into the tree. We don't use alloca() in our tree unless
it is entirely unavoidable (for example libc/*/exec.c), because any erroneous
size controlled by an attacker turns into a known-location object placement
in a very dangerous region. So use malloc() instead.


# 1.12 26-Nov-2018 reyk

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

branches: 1.8.2;
Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.13 10-Jan-2019 deraadt

an alloca() snuck into the tree. We don't use alloca() in our tree unless
it is entirely unavoidable (for example libc/*/exec.c), because any erroneous
size controlled by an attacker turns into a known-location object placement
in a very dangerous region. So use malloc() instead.


# 1.12 26-Nov-2018 reyk

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

branches: 1.8.2;
Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.12 26-Nov-2018 reyk

Move the {qcow2,raw} create functions from vmctl into vmd/vio{qcow2,raw}.c

This way they are in the appropriate place and code can be shared with vmd.

Ok ori@ mlarkin@ ccardenas@


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

branches: 1.8.2;
Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.11 24-Nov-2018 ori

Improve error handling and logging in qcow2

This turns most warn + returns that should never happen into hard failures,
and improves the user directed error messages.

ok @mlarkin, @reyk


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.10 24-Oct-2018 ori

Fix qcow2 disk images for data sizes greater than 4 gigs.

We used to truncate the disk end by anding it with a 32 bit value.
The 32 bit value was not sign extended, which causes the disk size
to wrap at 4 gigabytes:

disk->end = (disk->end + disk->clustersz - 1) & ~(disk->clustersz - 1);

This change converts the clustersz to an off_t in order to remove the
class of errors by avoiding type conversions entirely.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


# 1.9 19-Oct-2018 reyk

Add support to create and convert disk images from existing images

The -i option to vmctl create (eg. vmctl create output.qcow2 -i input.img)
lets you create a new image from an input file and convert it if it is a
different format. This allows to convert qcow2 images from raw images,
raw from qcow2, or even qcow2 from qcow2 and raw from raw to re-optimize
the disk.

This re-uses Ori's vioqcow2.c from vmd by reaching into it and
compiling it in. The API has been adjust to be used from both vmctl
and vmd accordingly.

OK mlarkin@


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.


Revision tags: OPENBSD_6_4_BASE
# 1.8 08-Oct-2018 reyk

Add support for qcow2 base images (external snapshots).

This works is from Ori Bernstein, committing on his behalf:

Add support to vmd for external snapshots. That is, snapshots that are
derived from a base image. Data lookups start in the derived image,
and if the derived image does not contain some data, the search
proceeds ot the base image. Multiple derived images may exist off of
a single base image.

A limitation of this format is that modifying the base image will
corrupt the derived image.

This change also adds support for creating disk derived disk images to
vmctl. To use it:

vmctl create derived.qcow2 -s 16G -b base.qcow2

From Ori Bernstein
OK mlarkin@ reyk@


# 1.7 01-Oct-2018 reyk

Fix potential double-free in error path

qc2_open() calls qc2_close() on error which already frees diskp.

OK ccardenas@


# 1.6 01-Oct-2018 reyk

Try to derive the qcow2 file format from an image file automatically.

This makes the "-d qcow2:" and "format qcow" arguments optional as vmctl
and vmd will read the magic bytes at the beginning of a file to guess if
it is a raw or a qcow image file.

The "vmctl create" command has been changed by removing the -f qcow2 option
and replacing it with the same syntax as -d: "vmctl create qcow2:foo.img".
In a slightly ununixy but intended way, the create command now also
considers the file extension for the format as "vmctl create foo.qcow2"
creates a qcow2 disk and not a raw image file.

Ok mlarkin@ (and ccardenas@ on an earlier version of the diff)


# 1.5 28-Sep-2018 reyk

Support vmd-internal's vmboot with qcow2 disk images.

OK mlarkin@


# 1.4 28-Sep-2018 reyk

Compress qcow2 open debug messages into a single line

Please avoid tabs and excessive multi-line information with log_debug
as it also goes to syslog.

No functional change.


# 1.3 19-Sep-2018 ccardenas

Various clean up items for disks.

- qcow2: general cleanup
- vioraw: check malloc
- virtio: add function to sync disks
- vm: call virtio_shutdown to sync disks when vm is finished executing

Thanks to Ori Bernstein.

Ok miko@


# 1.2 11-Sep-2018 ccardenas

Fail fast when we are unable to determine disk format.

While here, minor cleanup on logging.


# 1.1 09-Sep-2018 ccardenas

Add initial qcow2 image support.

Users are able to declare disk images as 'raw' or 'qcow2' using either
vmctl and vm.conf. The default disk image format is 'raw' if not specified.

Examples of using disk format:

vmctl start bsd -Lc -r cd64.iso -d qcow2:current.qc2
or
vmctl start bsd -Lc -r cd64.iso -d raw:current.raw
is equivalent to
vmctl start bsd -Lc -r cd64.iso -d current.raw

in vm.conf
vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.qc2" format "qcow2"
interface { switch "external" }
}

or

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw" format "raw"
interface { switch "external" }
}

is equivlanet to

vm "current" {
disable
memory 2G
disk "/home/user/vmm/current.raw"
interface { switch "external" }
}

Tested by many.

Big Thanks to Ori Bernstein.