README
1VMKIT README
2============
3
4The monitor needs two files to run. A GRUB stage 2 image and a disk image with
5the guest installed. The disk image hast to be readable by GRUB, of course. The
6files are loaded via NFS and their location can be configured in vmkitmon.c.
7
8NOTE I: VMkit does only support a trivial BIOS disk. No real virtual disk
9controller is implemented. Therefore, the guest disk image has to be configured
10to load everything relevant into RAM before invoking the guest kernel.
11
12NOTE II: VMkit does not compile in cmake debug mode atm. The RBP register is
13accessed when saving and restoring the guest's state. The standard
14"RelWithDebInfo" works.
15
16The construction of the disk image:
17
18The image is composed of a partition table and a partition (hd0,0) with an ext3
19filesystem containing the GRUB menu.lst (in the boot/grub directory as
20specified in the GRUB stage 2 image), the Linux kernel, initramfs, and other
21pertinent files, which can be accessed in a relatively simple manner by using
22loop devices in Linux:
23
24losetup /dev/loop0 system-bench.img
25losetup -o 32256 /dev/loop1 /dev/loop0
26mk -t ext3 /dev/loop1
27mount -t ext3 /dev/loop1 /mnt/example
28
29The offset 32256, the start sector of the partition multiplied by the sector
30size (in bytes), is obtained by using a disk partitioning utility/application
31such as fdisk or cfdisk. In this case, the start sector is 63 and the sector
32size is 512 bytes.
33
34
35GRUB
36====
37
38The GRUB stage 2 used is compiled with a compiled in menu list
39(--enable-preset-menu configure option) of:
40
41serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
42terminal --timeout=0 serial
43default 0
44timeout 0
45title Load config from (hd0,0)
46configfile (hd0,0)/boot/grub/menu.lst
47
48The terminal and serial part is important. Otherwise GRUB will try to access the
49video console which will result in unhandled BIOS calls.
50
51The menu list that I used (on sbrinz1 and sbrinz2) was:
52
53timeout 0
54title Barrelfish
55root (nd)
56kernel /sandrinr/kernel loglevel=3 logmask=0
57module /sandrinr/init
58module /sandrinr/mem_serv
59module /sandrinr/monitor bootcpus=0-1
60module /sandrinr/chips
61module /sandrinr/serial
62module /sandrinr/lpc_timer
63module /sandrinr/skb
64module /sandrinr/pci
65module /sandrinr/e1000n core=1
66module /sandrinr/vmkitmon
67
68
69Linux
70=====
71
72Linux does not believe the memory map reported by VMkit (INT 15, AX 0xe820).
73It is pretty primitive and should possibly be altered to reflect something
74similar as reported by QEMU.
75As a workaround, the Linux kernel parameters "mem=2G memmap=640K@0 memmap=1999M@1M"
76can be used.
77
78The Linux kernel parameters used for the test system are:
79"edd=off noapic nolapic noirqbalance acpi=off pci=off noisapnp nousb
80mem=2G memmap=640K@0 memmap=1999M@1M console=ttyS0,115200 1"
81It is possible that not everything is necessary. I just put everything in there
82i found to disable unsupported core subsystems.
83
README_VMX
1=====================================================================================
2VMkit monitor with Intel VMX virtualization extensions
3=====================================================================================
4
5IMPLEMENTATION
6
7Most information associated with the monitor implementation and the virtual hardware
8that is used is contained in the thesis "VMkit A lightweight hypervisor library for
9Barrelfish."
10
11Other details (especially those affiliated with Intel extensions):
12
13Two configuration variables, config_svm and config_arrakismon, in hake/Config.hs
14determine what hardware virtualization extensions are used and what types of guest
15VMs can be used, respectively. This should be changed later to allow both Arrakis
16domains and normal guests to exist simultaneously. Since changes in guest.c for Intel
17VMX extensions make use of config_svm immensely, an abstraction library could be
18introduced in the future to make the monitor code easier to read.
19
20Most significant changes when using Intel VMX extensions:
21
22-The VMCS of each guest-VM are accessed and modified by the privileged instructions
23vmread and vmwrite. This currently requires issuing syscalls that execute these
24corresponding instructions. Other privileged instructions are required to be
25executed when launching/resuming each guest.
26
27-The initialization of the VMCS is currently performed in kernel-space when the DCB
28is setup.
29
30-A CR0 shadow must be used to cause a VM-exit and dispatch the monitor when
31protected-mode is disabled.
32
33-The interception of certain events (such as executing specific instructions) is
34managed by setting specific VMX controls.
35
36-When interrupts are processed, the most prioritized interrupt is injected (by
37virq_handler in guest.c) only when the guest is willing to accept interrupts: changes
38had to be made to lpc.c to assert a virtual interrupt only when the function
39virq_accepting returns true.
40
41When there is a interrupt pending, but the guest is unwilling to accept them, an
42interrupt-window can be created at this time to cause a VM-exit as soon as possible.
43However, this is not yet implemented.
44
45-Prior to VM-entry, the host state that is restored on VM-exit must be manually
46saved into the VMCS. The function vmx_set_host_state in vmx_vmkit.c executes a series
47of vmwrite instructions to accomplish this.
48
49-Because rigorous checks are performed by the processor on the guest-state area,
50host-state area, equivalent checks (in vmx_check.s) occur before the guest is
51launched for the first time; this can assist in determining the exact test(s) the
52processor fails on.
53
54=====================================================================================
55
56ASSUMPTIONS AND CAVEATS
57
58The assumptions and caveats from the original implementation still hold. For example:
59- The existence of only one guest that makes use of only one vCPU is supported.
60
61- Each guest-VM is allocated 2 GB of memory. The amount, however, can easily be
62changed in guest.c.
63
64- At most eleven different MSRs can be set and accessed by the guest.
65...
66
67More assumptions when using Intel extensions:
68
69-In the interrupt handler in the real-mode emulator, software interrupts are always
70intercepted.
71
72-If a VM-exit transpires because of a VM-entry failure due to invalid guest state,
73it is assumed that an invalid guest GS limit with a value of 0x10ffef is the cause.
74With the Linux version that was used to test the implementation, this type of VM-exit
75occurs only once.
76
77Note that there may be other problems/issues that we are yet unaware of.
78
79=====================================================================================
80
81TESTING
82
83The script tools/harness/scalebench.py can be run with tools/harness/tests/vmkit.py
84as an argument. The test successfully terminates when the string "bash-4.0#" is
85printed to console output.
86
87Changes might need to be made to the modules that are loaded; these are specified in
88tools/harness/barrelfish.py and tools/harness/tests/vmkit.py.
89
90=====================================================================================
91
92TODO
93
94- Supporting other VM-exit reasons and implementing corresponding handlers in the
95monitor or in kernel space, if necessary.
96
97- Additional checks (in vmx_checks.s) for when the guest is in virtual-8086 mode,
98when PAE paging is in use, and on several fields comprising guest non-register state.
99
100- Using VM-entry event injection for reflecting guest (both benign and contributory)
101exceptions, injecting non-maskable interrupts (NMIs), and other events.
102
103- When a VM-exit occurs due to an external interrupt and the guest is unwilling to
104accept interrupts, allow the option of creating an interrupt window by setting a
105VM-execution control to cause a VM-exit to allow the injection of a pending interrupt.
106A NMI-window could also be created for virtual NMIs.
107
108- Allowing support for multi-core VMs. One should be able to simply port the VMkit
109modifications presented in the thesis (http://www.barrelfish.org/ma-humbell-final.pdf)
110by Lukas Humbel when (and if) they are committed to Barrelfish.
111
112- Permitting more than one VMCS and the moving of a VMCS between processors.
113
114- APIC virtualization
115
116...
117
118=====================================================================================
119
120DOCUMENTATION
121
122Implementation is based on Intel 64 and IA-32 Architectures Software Developer's
123Manual Volume 3C: System Programming Guide, Part 3 (June 2014) -
124
125http://www.intel.com/content/dam/www/public/us/en/documents/manuals/
12664-ia-32-architectures-software-developer-vol-3c-part-3-manual.pdf
127