#
365777 |
|
15-Sep-2020 |
emaste |
MFC r365775: bhyve: do not permit write access to VMCB / VMCS
Reported by: Patrick Mooney Submitted by: jhb Security: CVE-2020-24718
|
#
360432 |
|
28-Apr-2020 |
grehan |
MFC r358848: Untangle TPR shadowing and APIC virtualization.
This speeds up Windows guests tremendously.
The patch does: Add a new tuneable 'hw.vmm.vmx.use_tpr_shadowing' to disable TLP shadowing. Also add 'hw.vmm.vmx.cap.tpr_shadowing' to be able to query if TPR shadowing is used.
Detach the initialization of TPR shadowing from the initialization of APIC virtualization. APIC virtualization still needs TPR shadowing, but not vice versa. Any CPU that supports APIC virtualization should also support TPR shadowing.
When TPR shadowing is used, the APIC page of each vCPU is written to the VMCS_VIRTUAL_APIC field of the VMCS so that the CPU can write directly to the page without intercept.
On vm exit, vlapic_update_ppr() is called to update the PPR.
Submitted by: Yamagi Burmeister Reviewed by: grehan Differential Revision: https://reviews.freebsd.org/D22942 Approved by: bz (mentor) Tested by: Jason Tubnor
|
#
351753 |
|
03-Sep-2019 |
emaste |
MFC r350492: vmx: use C99 bool, not boolean_t
Bhyve's vmm is a self-contained modern component and thus a good candidate for use of C99 types.
Sponsored by: The FreeBSD Foundation
|
#
348271 |
|
25-May-2019 |
rgrimes |
MFC: r346714: Add accessor function for vm->maxcpus
Replace most VM_MAXCPU constant useses with an accessor function to vm->maxcpus which for now is initialized and kept at the value of VM_MAXCPUS.
This is a rework of Fabian Freyer (fabian.freyer_physik.tu-berlin.de) work from D10070 to adjust it for the cpu topology changes that occured in r332298
Approved by: re (kib)
|
#
347435 |
|
10-May-2019 |
jhb |
MFC 338957: Handle a guest executing a vm instruction by trapping and raising an undefined instruction exception. Previously we would exit the guest, however an unprivileged user could execute these.
|
#
347416 |
|
10-May-2019 |
jhb |
MFC 332479: Add SDT probes to vmexit on Intel.
|
#
347074 |
|
04-May-2019 |
jhb |
MFC 330615: Fix a lock recursion introduced in r327065.
|
#
347073 |
|
04-May-2019 |
jhb |
MFC 327065: Recognize a pending virtual interrupt while emulating the halt instruction.
|
#
346978 |
|
01-May-2019 |
jhb |
MFC 344711: Fix missed posted interrupts in VT-x in bhyve.
When a vCPU is HLTed, interrupts with a priority below the processor priority (PPR) should not resume the vCPU while interrupts at or above the PPR should. With posted interrupts, bhyve maintains a bitmap of pending interrupts in PIR descriptor along with a single 'pending' bit. This bit is checked by a CPU running in guest mode at various places to determine if it should be checked. In addition, another CPU can force a CPU in guest mode to check for pending interrupts by sending an IPI to a special IDT vector reserved for this purpose.
bhyve had a bug in that it would only notify a guest vCPU of an interrupt (e.g. by sending the special IPI or by resuming it if it was idle due to HLT) if an interrupt arrived that was higher priority than PPR and no interrupts were currently pending. This assumed that if the 'pending' bit was set, any needed notification was already in progress. However, if the first interrupt sent to a HLTed vCPU was lower priority than PPR and the second was higher than PPR, the first interrupt would set 'pending' but not notify the vCPU, and the second interrupt would not notify the vCPU because 'pending' was already set. To fix this, track the priority of pending interrupts in a separate per-vCPU bitmask and notify a vCPU anytime an interrupt arrives that is above PPR and higher than any previously-received interrupt.
This was found and debugged in the bhyve port to SmartOS maintained by Joyent. Relevant SmartOS bugs with more background:
https://smartos.org/bugview/OS-6829 https://smartos.org/bugview/OS-6930 https://smartos.org/bugview/OS-7354
|
#
343212 |
|
20-Jan-2019 |
kib |
MFC r343108: Trim whitespace at EoL, use tabs instead of spaces for indent.
PR: 235004
|
#
340609 |
|
19-Nov-2018 |
kib |
MFC r340487: Align IA32_ARCH_CAP MSR definitions and use with SDM rev. 068.
|
#
340545 |
|
18-Nov-2018 |
jhb |
MFC 339312,339364: Restore more descriptors during VM exits.
339312: Fully restore the GDTR, IDTR, and LDTR after VT-x VM exits.
The VT-x VMCS only stores the base address of the GDTR and IDTR. As a result, VM exits use a fixed limit of 0xffff for the host GDTR and IDTR losing the smaller limits set in when the initial GDT is loaded on each CPU during boot. Explicitly save and restore the full GDTR and IDTR contents around VM entries and exits to restore the correct limit.
Similarly, explicitly save and restore the LDT selector. VM exits always clear the host LDTR as if the LDT was loaded with a NULL selector and a userspace hypervisor is probably using a NULL selector anyway, but save and restore the LDT explicitly just to be safe.
339364: Reload the LDT selector after an AMD-v #VMEXIT.
cpu_switch() always reloads the LDT, so this can only affect the hypervisor process itself. Fix this by explicitly reloading the host LDT selector after each #VMEXIT. The stock bhyve process on FreeBSD never uses a custom LDT, so this change is cosmetic.
PR: 230773
|
#
338691 |
|
14-Sep-2018 |
jhb |
MFC 332454,334009,334122: Various fixes for x86 debug exceptions.
332454: Fix PSL_T inheritance on exec for x86.
The miscellaneous x86 sysent->sv_setregs() implementations tried to migrate PSL_T from the previous program to the new executed one, but they evaluated regs->tf_eflags after the whole regs structure was bzeroed. Make this functional by saving PSL_T value before zeroing.
Note that if the debugger is not attached, executing the first instruction in the new program with PSL_T set results in SIGTRAP, and since all intercepted signals are reset to default dispostion on exec(2), this means that non-debugged process gets killed immediately if PSL_T is inherited. In particular, since suid images drop P_TRACED, attempt to set PSL_T for execution of such program would kill the process.
Another issue with userspace PSL_T handling is that it is reset by trap(). It is reasonable to clear PSL_T when entering SIGTRAP handler, to allow the signal to be handled without recursion or delivery of blocked fault. But it is not reasonable to return back to the normal flow with PSL_T cleared. This is too late to change, I think.
334009: Cleanups related to debug exceptions on x86.
- Add constants for fields in DR6 and the reserved fields in DR7. Use these constants instead of magic numbers in most places that use DR6 and DR7. - Refer to T_TRCTRAP as "debug exception" rather than a "trace trap" as it is not just for trace exceptions. - Always read DR6 for debug exceptions and only clear TF in the flags register for user exceptions where DR6.BS is set. - Clear DR6 before returning from a debug exception handler as recommended by the SDM dating all the way back to the 386. This allows debuggers to determine the cause of each exception. For kernel traps, clear DR6 in the T_TRCTRAP case and pass DR6 by value to other parts of the handler (namely, user_dbreg_trap()). For user traps, wait until after trapsignal to clear DR6 so that userland debuggers can read DR6 via PT_GETDBREGS while the thread is stopped in trapsignal().
334122: x86: stop unconditionally clearing PSL_T on the trace trap.
We certainly should clear PSL_T when calling the SIGTRAP signal handler, which is already done by all x86 sendsig(9) ABI code. On the other hand, there is no obvious reason why PSL_T needs to be cleared when returning from the signal handler. For instance, Linux allows userspace to set PSL_T and keep tracing enabled for the desired period. There are userspace programs which would use PSL_T if we make it possible, for instance sbcl.
Remember if PSL_T was set by PT_STEP or PT_SETSTEP by mean of TDB_STEP flag, and only clear it when the flag is set.
|
#
338427 |
|
02-Sep-2018 |
kib |
MFC r338068, r338113: Update L1TF workaround to sustain L1D pollution from NMI.
|
#
337794 |
|
14-Aug-2018 |
kib |
MFC r337785: Provide part of the mitigation for L1TF-VMM.
Security: CVE-2018-3646 Approved by: so (insta-MFC)
|
#
331722 |
|
29-Mar-2018 |
eadler |
Revert r330897:
This was intended to be a non-functional change. It wasn't. The commit message was thus wrong. In addition it broke arm, and merged crypto related code.
Revert with prejudice.
This revert skips files touched in r316370 since that commit was since MFCed. This revert also skips files that require $FreeBSD$ property changes.
Thank you to those who helped me get out of this mess including but not limited to gonzo, kevans, rgrimes.
Requested by: gjb (re)
|
#
330897 |
|
14-Mar-2018 |
eadler |
Partial merge of the SPDX changes
These changes are incomplete but are making it difficult to determine what other changes can/should be merged.
No objections from: pfg
|
#
330623 |
|
07-Mar-2018 |
jhb |
MFC 328102: Save and restore guest debug registers.
Currently most of the debug registers are not saved and restored during VM transitions allowing guest and host debug register values to leak into the opposite context. One result is that hardware watchpoints do not work reliably within a guest under VT-x.
Due to differences in SVM and VT-x, slightly different approaches are used.
For VT-x:
- Enable debug register save/restore for VM entry/exit in the VMCS for DR7 and MSR_DEBUGCTL. - Explicitly save DR0-3,6 of the guest. - Explicitly save DR0-3,6-7, MSR_DEBUGCTL, and the trap flag from %rflags for the host. Note that because DR6 is "software" managed and not stored in the VMCS a kernel debugger which single steps through VM entry could corrupt the guest DR6 (since a single step trap taken after loading the guest DR6 could alter the DR6 register). To avoid this, explicitly disable single-stepping via the trace flag before loading the guest DR6. A determined debugger could still defeat this by setting a breakpoint after the guest DR6 was loaded and then single-stepping.
For SVM: - Enable debug register caching in the VMCB for DR6/DR7. - Explicitly save DR0-3 of the guest. - Explicitly save DR0-3,6-7, and MSR_DEBUGCTL for the host. Since SVM saves the guest DR6 in the VMCB, the race with single-stepping described for VT-x does not exist.
For both platforms, expose all of the guest DRx values via --get-drX and --set-drX flags to bhyvectl.
|
#
329462 |
|
17-Feb-2018 |
kib |
MFC r328083,328096,328116,328119,328120,328128,328135,328153,328157, 328166,328177,328199,328202,328205,328468,328470,328624,328625,328627, 328628,329214,329297,329365:
Meltdown mitigation by PTI, PCID optimization of PTI, and kernel use of IBRS for some mitigations of Spectre.
Tested by: emaste, Arshan Khanifar <arshankhanifar@gmail.com> Discussed with: jkim Sponsored by: The FreeBSD Foundation
|
#
302408 |
|
07-Jul-2016 |
gjb |
Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle. Prune svn:mergeinfo from the new branch, as nothing has been merged here.
Additional commits post-branch will follow.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation |
#
284539 |
|
18-Jun-2015 |
neel |
Restructure memory allocation in bhyve to support "devmem".
devmem is used to represent MMIO devices like the boot ROM or a VESA framebuffer where doing a trap-and-emulate for every access is impractical. devmem is a hybrid of system memory (sysmem) and emulated device models.
devmem is mapped in the guest address space via nested page tables similar to sysmem. However the address range where devmem is mapped may be changed by the guest at runtime (e.g. by reprogramming a PCI BAR). Also devmem is usually mapped RO or RW as compared to RWX mappings for sysmem.
Each devmem segment is named (e.g. "bootrom") and this name is used to create a device node for the devmem segment (e.g. /dev/vmm/testvm.bootrom). The device node supports mmap(2) and this decouples the host mapping of devmem from its mapping in the guest address space (which can change).
Reviewed by: tychon Discussed with: grehan Differential Revision: https://reviews.freebsd.org/D2762 MFC after: 4 weeks
|
#
284174 |
|
08-Jun-2015 |
tychon |
Support guest writes to the TSC by enabling the "use TSC offsetting" execution control and writing the difference between the host TSC and the guest TSC into the TSC offset in the VMCS upon encountering a write.
Reviewed by: neel
|
#
283657 |
|
28-May-2015 |
neel |
Fix non-deterministic delays when accessing a vcpu that was in "running" or "sleeping" state. This is done by forcing the vcpu to transition to "idle" by returning to userspace with an exit code of VM_EXITCODE_REQIDLE.
MFC after: 2 weeks
|
#
283293 |
|
22-May-2015 |
neel |
Don't rely on the 'VM-exit instruction length' field in the VMCS to always have an accurate length on an EPT violation. This is not needed by the instruction decoding code because it also has to work with AMD/SVM that does not provide a valid instruction length on a Nested Page Fault.
In collaboration with: Leon Dang (ldang@nahannisys.com) Discussed with: grehan MFC after: 1 week
|
#
280447 |
|
24-Mar-2015 |
tychon |
When fetching an instruction in non-64bit mode, consider the value of the code segment base address.
Also if an instruction doesn't support a mod R/M (modRM) byte, don't be concerned if the CPU is in real mode.
Reviewed by: neel
|
#
279971 |
|
14-Mar-2015 |
neel |
Use lapic_ipi_alloc() to dynamically allocate IPI slots needed by bhyve when vmm.ko is loaded.
Also relocate the 'justreturn' IPI handler to be alongside all other handlers.
Requested by: kib
|
#
279228 |
|
24-Feb-2015 |
neel |
Always emulate MSR_PAT on Intel processors and don't rely on PAT save/restore capability of VT-x. This lets bhyve run nested in older VMware versions that don't support the PAT save/restore capability.
Note that the actual value programmed by the guest in MSR_PAT is irrelevant because bhyve sets the 'Ignore PAT' bit in the nested PTE.
Reported by: marcel Tested by: Leon Dang (ldang@nahannisys.com) Sponsored by: Nahanni Systems MFC after: 2 weeks
|
#
277310 |
|
18-Jan-2015 |
neel |
Simplify instruction restart logic in bhyve.
Keep track of the next instruction to be executed by the vcpu as 'nextrip'. As a result the VM_RUN ioctl no longer takes the %rip where a vcpu should start execution.
Also, instruction restart happens implicitly via 'vm_inject_exception()' or explicitly via 'vm_restart_instruction()'. The APIs behave identically in both kernel and userspace contexts. The main beneficiary is the instruction emulation code that executes in both contexts.
bhyve(8) VM exit handlers now treat 'vmexit->rip' and 'vmexit->inst_length' as readonly: - Restarting an instruction is now done by calling 'vm_restart_instruction()' as opposed to setting 'vmexit->inst_length' to 0 (e.g. emulate_inout()) - Resuming vcpu at an arbitrary %rip is now done by setting VM_REG_GUEST_RIP as opposed to changing 'vmexit->rip' (e.g. vmexit_task_switch())
Differential Revision: https://reviews.freebsd.org/D1526 Reviewed by: grehan MFC after: 2 weeks
|
#
277149 |
|
13-Jan-2015 |
neel |
'struct vm_exception' was intended to be used only as the collateral for the VM_INJECT_EXCEPTION ioctl. However it morphed into other uses like keeping track pending exceptions for a vcpu. This in turn causes confusion because some fields in 'struct vm_exception' like 'vcpuid' make sense only in the ioctl context. It also makes it harder to add or remove structure fields.
Fix this by using 'struct vm_exception' only to communicate information from userspace to vmm.ko when injecting an exception.
Also, add a field 'restart_instruction' to 'struct vm_exception'. This field is set to '1' for exceptions where the faulting instruction is restarted after the exception is handled.
MFC after: 1 week
|
#
276763 |
|
06-Jan-2015 |
neel |
Clear blocking due to STI or MOV SS in the hypervisor when an instruction is emulated or when the vcpu incurs an exception. This matches the CPU behavior.
Remove special case code in HLT processing that was clearing the interrupt shadow. This is now redundant because the interrupt shadow is always cleared when the vcpu is resumed after an instruction is emulated.
Reported by: David Reed (david.reed@tidalscale.com) MFC after: 2 weeks
|
#
276098 |
|
23-Dec-2014 |
neel |
Allow ktr(4) tracing of all guest exceptions via the tunable "hw.vmm.trace_guest_exceptions". To enable this feature set the tunable to "1" before loading vmm.ko.
Tracing the guest exceptions can be useful when debugging guest triple faults.
Note that there is a performance impact when exception tracing is enabled since every exception will now trigger a VM-exit.
Also, handle machine check exceptions that happen during guest execution by vectoring to the host's machine check handler via "int $18".
Discussed with: grehan MFC after: 2 weeks
|
#
273375 |
|
21-Oct-2014 |
neel |
Merge projects/bhyve_svm into HEAD.
After this change bhyve supports AMD processors with the SVM/AMD-V hardware extensions.
More details available here: https://lists.freebsd.org/pipermail/freebsd-virtualization/2014-October/002905.html
Submitted by: Anish Gupta (akgupt3@gmail.com) Tested by: Benjamin Perrault (ben.perrault@gmail.com) Tested by: Willem Jan Withagen (wjw@digiware.nl)
|
#
272670 |
|
06-Oct-2014 |
neel |
Inject #UD into the guest when it executes either 'MONITOR' or 'MWAIT'.
The hypervisor hides the MONITOR/MWAIT capability by unconditionally setting CPUID.01H:ECX[3] to 0 so the guest should not expect these instructions to be present anyways.
Discussed with: grehan
|
#
272395 |
|
02-Oct-2014 |
neel |
Get rid of code that dealt with the hardware not being able to save/restore the PAT MSR on guest exit/entry. This workaround was done for a beta release of VMware Fusion 5 but is no longer needed in later versions.
All Intel CPUs since Nehalem have supported saving and restoring MSR_PAT in the VM exit and entry controls.
Discussed with: grehan
|
#
271890 |
|
20-Sep-2014 |
neel |
MSR_KGSBASE is no longer saved and restored from the guest MSR save area. This behavior was changed in r271888 so update the comment block to reflect this.
MSR_KGSBASE is accessible from the guest without triggering a VM-exit. The permission bitmap for MSR_KGSBASE is modified by vmx_msr_guest_init() so get rid of redundant code in vmx_vminit().
|
#
271888 |
|
20-Sep-2014 |
neel |
Restructure the MSR handling so it is entirely handled by processor-specific code. There are only a handful of MSRs common between the two so there isn't too much duplicate functionality.
The VT-x code has the following types of MSRs:
- MSRs that are unconditionally saved/restored on every guest/host context switch (e.g., MSR_GSBASE).
- MSRs that are restored to guest values on entry to vmx_run() and saved before returning. This is an optimization for MSRs that are not used in host kernel context (e.g., MSR_KGSBASE).
- MSRs that are emulated and every access by the guest causes a trap into the hypervisor (e.g., MSR_IA32_MISC_ENABLE).
Reviewed by: grehan
|
#
271451 |
|
12-Sep-2014 |
neel |
Optimize the common case of injecting an interrupt into a vcpu after a HLT by explicitly moving it out of the interrupt shadow. The hypervisor is done "executing" the HLT and by definition this moves the vcpu out of the 1-instruction interrupt shadow.
Prior to this change the interrupt would be held pending because the VMCS guest-interruptibility-state would indicate that "blocking by STI" was in effect. This resulted in an unnecessary round trip into the guest before the pending interrupt could be injected.
Reviewed by: grehan
|
#
269281 |
|
29-Jul-2014 |
jhb |
- Output a summary of optional VT-x features in dmesg similar to CPU features. If bootverbose is enabled, a detailed list is provided; otherwise, a single-line summary is displayed. - Add read-only sysctls for optional VT-x capabilities used by bhyve under a new hw.vmm.vmx.cap node. Move a few exiting sysctls that indicate the presence of optional capabilities under this node.
CR: https://phabric.freebsd.org/D498 Reviewed by: grehan, neel MFC after: 1 week
|
#
269109 |
|
26-Jul-2014 |
neel |
If a vcpu has issued a HLT instruction with interrupts disabled then it sleeps forever in vm_handle_hlt().
This is usually not an issue as long as one of the other vcpus properly resets or powers off the virtual machine. However, if the bhyve(8) process is killed with a signal the halted vcpu cannot be woken up because it's sleep cannot be interrupted.
Fix this by waking up periodically and returning from vm_handle_hlt() if TDF_ASTPENDING is set.
Reported by: Leon Dang Sponsored by: Nahanni Systems
|
#
268922 |
|
20-Jul-2014 |
neel |
Fix build without INVARIANTS defined by getting rid of unused variable 'exc'.
Reported by: adrian, stefanf
|
#
268889 |
|
19-Jul-2014 |
neel |
Handle nested exceptions in bhyve.
A nested exception condition arises when a second exception is triggered while delivering the first exception. Most nested exceptions can be handled serially but some are converted into a double fault. If an exception is generated during delivery of a double fault then the virtual machine shuts down as a result of a triple fault.
vm_exit_intinfo() is used to record that a VM-exit happened while an event was being delivered through the IDT. If an exception is triggered while handling the VM-exit it will be treated like a nested exception.
vm_entry_intinfo() is used by processor-specific code to get the event to be injected into the guest on the next VM-entry. This function is responsible for deciding the disposition of nested exceptions.
|
#
268777 |
|
16-Jul-2014 |
neel |
Add emulation for legacy x86 task switching mechanism.
FreeBSD/i386 uses task switching to handle double fault exceptions and this change enables that to work.
Reported by: glebius
|
#
268701 |
|
15-Jul-2014 |
neel |
Add support for operand size and address size override prefixes in bhyve's instruction emulation [1].
Fix bug in emulation of opcode 0x8A where the destination is a legacy high byte register and the guest vcpu is in 32-bit mode. Prior to this change instead of modifying %ah, %bh, %ch or %dh the emulation would end up modifying %spl, %bpl, %sil or %dil instead.
Add support for moffsets by treating it as a 2, 4 or 8 byte immediate value during instruction decoding.
Fix bug in verify_gla() where the linear address computed after decoding the instruction was not being truncated to the effective address size [2].
Tested by: Leon Dang [1] Reported by: Peter Grehan [2] Sponsored by: Nahanni Systems
|
#
268428 |
|
08-Jul-2014 |
neel |
Accurately identify the vcpu's operating mode as 64-bit, compatibility, protected or real.
|
#
268427 |
|
08-Jul-2014 |
neel |
Invalidate guest TLB mappings as a side-effect of its CR3 being updated.
This is a pre-requisite for task switch emulation since the CR3 is loaded from the new TSS.
|
#
267558 |
|
16-Jun-2014 |
tychon |
Bring an overly enthusiastic KASSERT inline with the Intel SDM.
Reviewed by: neel
|
#
267330 |
|
10-Jun-2014 |
neel |
Add helper functions to populate VM exit information for rendezvous and astpending exits. This is to reduce code duplication between VT-x and SVM implementations.
|
#
267311 |
|
09-Jun-2014 |
neel |
Turn on interrupt window exiting unconditionally when an ExtINT is being injected into the guest. This allows the hypervisor to inject another ExtINT or APIC vector as soon as the guest is able to process interrupts.
This change is not to address any correctness issue but to guarantee that any pending APIC vector that was preempted by the ExtINT will be injected as soon as possible. Prior to this change such pending interrupts could be delayed until the next VM exit.
|
#
267300 |
|
09-Jun-2014 |
neel |
Add reserved bit checking when doing %CR8 emulation and inject #GP if required.
Pointed out by: grehan Reviewed by: tychon
|
#
267178 |
|
06-Jun-2014 |
tychon |
Support guest accesses to %cr8.
Reviewed by: neel
|
#
266910 |
|
30-May-2014 |
tychon |
If VMX isn't enabled so long as the lock bit isn't set yet in MSR IA32_FEATURE_CONTROL it still can be.
Approved by: grehan (co-mentor)
|
#
266765 |
|
27-May-2014 |
jhb |
- Rework the XSAVE/XRSTOR emulation to only expose XCR0 features to the guest for which the rules regarding xsetbv emulation are known. In particular future extensions like AVX-512 have interdependencies among feature bits that could allow a guest to trigger a GP# in the host with the current approach of allowing anything the host supports. - Add proper checking of Intel MPX and AVX-512 XSAVE features in the xsetbv emulation and allow these features to be exposed to the guest if they are enabled in the host. - Expose a subset of known-safe features from leaf 0 of the structured extended features to guests if they are supported on the host including RDFSBASE/RDGSBASE, BMI1/2, AVX2, AVX-512, HLE, ERMS, and RTM. Aside from AVX-512, these features are all new instructions available for use in ring 3 with no additional hypervisor changes needed.
Reviewed by: neel
|
#
266641 |
|
24-May-2014 |
neel |
Do the linear address calculation for the ins/outs emulation using a new API function 'vie_calculate_gla()'.
While the current implementation is simplistic it forms the basis of doing segmentation checks if the guest is in 32-bit protected mode.
|
#
266627 |
|
24-May-2014 |
neel |
Consolidate all the information needed by the guest page table walker into 'struct vm_guest_paging'.
Check for canonical addressing in vmm_gla2gpa() and inject a protection fault into the guest if a violation is detected.
If the page table walk is restarted in vmm_gla2gpa() then reset 'ptpphys' to point to the root of the page tables.
|
#
266626 |
|
24-May-2014 |
neel |
When injecting a page fault into the guest also update the guest's %cr2 to indicate the faulting linear address.
If the guest PML4 entry has the PG_PS bit set then inject a page fault into the guest with the PGEX_RSV bit set in the error_code.
Get rid of redundant checks for the PG_RW violations when walking the page tables.
|
#
266573 |
|
23-May-2014 |
neel |
Add emulation of the "outsb" instruction. NetBSD guests use this to write to the UART FIFO.
The emulation is constrained in a number of ways: 64-bit only, doesn't check for all exception conditions, limited to i/o ports emulated in userspace.
Some of these constraints will be relaxed in followup commits.
Requested by: grehan Reviewed by: tychon (partially and a much earlier version)
|
#
266550 |
|
22-May-2014 |
neel |
Allow vmx_getdesc() and vmx_setdesc() to be called for a vcpu that is in the VCPU_RUNNING state. This will let the VMX exit handler inspect the vcpu's segment descriptors without having to exit the critical section.
|
#
266424 |
|
19-May-2014 |
neel |
Add PG_U (user/supervisor) checks when translating a guest linear address to a guest physical address.
PG_PS (page size) field is valid only in a PDE or a PDPTE so it is now checked only in non-terminal paging entries.
Ignore the upper 32-bits of the CR3 for PAE paging.
|
#
265114 |
|
30-Apr-2014 |
neel |
Ignore writes to microcode update MSR. This MSR is accessed by RHEL7 guest. Add KTR tracepoints to annotate wrmsr and rdmsr VM exits.
|
#
265062 |
|
28-Apr-2014 |
neel |
Allow a virtual machine to be forcibly reset or powered off. This is done by adding an argument to the VM_SUSPEND ioctl that specifies how the virtual machine should be suspended, viz. VM_SUSPEND_RESET or VM_SUSPEND_POWEROFF.
The disposition of VM_SUSPEND is also made available to the exit handler via the 'u.suspended' member of 'struct vm_exit'.
This capability is exposed via the '--force-reset' and '--force-poweroff' arguments to /usr/sbin/bhyvectl.
Discussed with: grehan@
|
#
264988 |
|
26-Apr-2014 |
neel |
A VMCS is always inactive when it exits the vmx_run() loop. Remove redundant code and the misleading comment that suggest otherwise.
Reviewed by: grehan@
|
#
264846 |
|
23-Apr-2014 |
grehan |
Allow the guest to read the TSC via MSR 0x10. NetBSD/amd64 does this, as does Linux on AMD CPUs.
Reviewed by: neel MFC after: 3 weeks
|
#
264324 |
|
10-Apr-2014 |
grehan |
Rework r264179.
- remove redundant code - remove erroneous setting of the error return in vmmdev_ioctl() - use style(9) initialization - in vmx_inject_pir(), document the race condition that the final conditional statement was detecting,
Tested with both gcc and clang builds.
Reviewed by: neel
|
#
264179 |
|
05-Apr-2014 |
imp |
Make the vmm code compile with gcc too. Not entirely sure things are correct for the pirbase test (since I'd have thought we'd need to do something even when the offset is 0 and that test looks like a misguided attempt to not use an uninitialized variable), but it is at least the same as today.
|
#
263780 |
|
26-Mar-2014 |
neel |
Add an ioctl to suspend a virtual machine (VM_SUSPEND). The ioctl can be called from any context i.e., it is not required to be called from a vcpu thread. The ioctl simply sets a state variable 'vm->suspend' to '1' and returns.
The vcpus inspect 'vm->suspend' in the run loop and if it is set to '1' the vcpu breaks out of the loop with a reason of 'VM_EXITCODE_SUSPENDED'. The suspend handler waits until all 'vm->active_cpus' have transitioned to 'vm->suspended_cpus' before returning to userspace.
Discussed with: grehan
|
#
263211 |
|
15-Mar-2014 |
tychon |
Fix a race wherein the source of an interrupt vector is wrongly attributed if an ExtINT arrives during interrupt injection.
Also, fix a spurious interrupt if the PIC tries to raise an interrupt before the outstanding one is accepted.
Finally, improve the PIC interrupt latency when another interrupt is raised immediately after the outstanding one is accepted by creating a vmexit rather than waiting for one to occur by happenstance.
Approved by: neel (co-mentor)
|
#
263035 |
|
11-Mar-2014 |
tychon |
Replace the userspace atpic stub with a more functional vmm.ko model.
New ioctls VM_ISA_ASSERT_IRQ, VM_ISA_DEASSERT_IRQ and VM_ISA_PULSE_IRQ can be used to manipulate the pic, and optionally the ioapic, pin state.
Reviewed by: jhb, neel Approved by: neel (co-mentor)
|
#
262624 |
|
28-Feb-2014 |
jhb |
Correct VMware capitalization.
Submitted by: joeld
|
#
262615 |
|
28-Feb-2014 |
jhb |
Workaround an apparent bug in VMWare Fusion's nested VT support where it triggers a VM exit with the exit reason of an external interrupt but without a valid interrupt set in the exit interrupt information.
Tested by: Michael Dexter Reviewed by: neel MFC after: 1 week
|
#
262506 |
|
25-Feb-2014 |
neel |
Queue pending exceptions in the 'struct vcpu' instead of directly updating the processor-specific VMCS or VMCB. The pending exception will be delivered right before entering the guest.
The order of event injection into the guest is: - hardware exception - NMI - maskable interrupt
In the Intel VT-x case, a pending NMI or interrupt will enable the interrupt window-exiting and inject it as soon as possible after the hardware exception is injected. Also since interrupts are inherently asynchronous, injecting them after the hardware exception should not affect correctness from the guest perspective.
Rename the unused ioctl VM_INJECT_EVENT to VM_INJECT_EXCEPTION and restrict it to only deliver x86 hardware exceptions. This new ioctl is now used to inject a protection fault when the guest accesses an unimplemented MSR.
Discussed with: grehan, jhb Reviewed by: jhb
|
#
262281 |
|
21-Feb-2014 |
neel |
Add support for x2APIC virtualization assist in Intel VT-x.
The vlapic.ops handler 'enable_x2apic_mode' is called when the vlapic mode is switched to x2APIC. The VT-x implementation of this handler turns off the APIC-access virtualization and enables the x2APIC virtualization in the VMCS.
The x2APIC virtualization is done by allowing guest read access to a subset of MSRs in the x2APIC range. In non-root operation the processor will satisfy an 'rdmsr' access to these MSRs by reading from the virtual APIC page instead.
The guest is also given write access to TPR, EOI and SELF_IPI MSRs which get special treatment in non-root operation. This is documented in the Intel SDM section titled "Virtualizing MSR-Based APIC Accesses".
Enforce that APIC-write and APIC-access VM-exits are handled only if APIC-access virtualization is enabled. The one exception to this is SELF_IPI virtualization which may result in an APIC-write VM-exit.
|
#
262144 |
|
18-Feb-2014 |
jhb |
A first pass at adding support for injecting hardware exceptions for emulated instructions. - Add helper routines to inject interrupt information for a hardware exception from the VM exit callback routines. - Use the new routines to inject GP and UD exceptions for invalid operations when emulating the xsetbv instruction. - Don't directly manipulate the entry interrupt info when a user event is injected. Instead, store the event info in the vmx state and only apply it during a VM entry if a hardware exception or NMI is not already pending. - While here, use HANDLED/UNHANDLED instead of 1/0 in a couple of routines.
Reviewed by: neel
|
#
261638 |
|
08-Feb-2014 |
jhb |
Add virtualized XSAVE support to bhyve which permits guests to use XSAVE and XSAVE-enabled features like AVX. - Store a per-cpu guest xcr0 register. When switching to the guest FPU state, switch to the guest xcr0 value. Note that the guest FPU state is saved and restored using the host's xcr0 value and xcr0 is saved/restored "inside" of saving/restoring the guest FPU state. - Handle VM exits for the xsetbv instruction by updating the guest xcr0. - Expose the XSAVE feature to the guest only if the host has enabled XSAVE, and only advertise XSAVE features enabled by the host to the guest. This ensures that the guest will only adjust FPU state that is a subset of the guest FPU state saved and restored by the host.
Reviewed by: grehan
|
#
261621 |
|
08-Feb-2014 |
neel |
Add a counter to differentiate between VM-exits due to nested paging faults and instruction emulation faults.
|
#
261617 |
|
08-Feb-2014 |
neel |
Fix a bug in the handling of VM-exits caused by non-maskable interrupts (NMI).
If a VM-exit is caused by an NMI then "blocking by NMI" is in effect on the CPU when the VM-exit is completed. No more NMIs will be recognized until the execution of an "iret".
Prior to this change the NMI handler was dispatched via a software interrupt with interrupts enabled. This meant that an interrupt could be recognized by the processor before the NMI handler completed its execution. The "iret" issued by the interrupt handler would then cause the "blocking by NMI" to be cleared prematurely.
This is now fixed by handling the NMI with interrupts disabled in addition to "blocking by NMI" already established by the VM-exit.
|
#
261504 |
|
05-Feb-2014 |
jhb |
Add support for FreeBSD/i386 guests under bhyve. - Similar to the hack for bootinfo32.c in userboot, define _MACHINE_ELF_WANT_32BIT in the load_elf32 file handlers in userboot. This allows userboot to load 32-bit kernels and modules. - Copy the SMAP generation code out of bootinfo64.c and into its own file so it can be shared with bootinfo32.c to pass an SMAP to the i386 kernel. - Use uint32_t instead of u_long when aligning module metadata in bootinfo32.c in userboot, as otherwise the metadata used 64-bit alignment which corrupted the layout. - Populate the basemem and extmem members of the bootinfo struct passed to 32-bit kernels. - Fix the 32-bit stack in userboot to start at the top of the stack instead of the bottom so that there is room to grow before the kernel switches to its own stack. - Push a fake return address onto the 32-bit stack in addition to the arguments normally passed to exec() in the loader. This return address is needed to convince recover_bootinfo() in the 32-bit locore code that it is being invoked from a "new" boot block. - Add a routine to libvmmapi to setup a 32-bit flat mode register state including a GDT and TSS that is able to start the i386 kernel and update bhyveload to use it when booting an i386 kernel. - Use the guest register state to determine the CPU's current instruction mode (32-bit vs 64-bit) and paging mode (flat, 32-bit, PAE, or long mode) in the instruction emulation code. Update the gla2gpa() routine used when fetching instructions to handle flat mode, 32-bit paging, and PAE paging in addition to long mode paging. Don't look for a REX prefix when the CPU is in 32-bit mode, and use the detected mode to enable the existing 32-bit mode code when decoding the mod r/m byte.
Reviewed by: grehan, neel MFC after: 1 month
|
#
261453 |
|
04-Feb-2014 |
neel |
Avoid doing unnecessary nested TLB invalidations.
Prior to this change the cached value of 'pm_eptgen' was tracked per-vcpu and per-hostcpu. In the degenerate case where 'N' vcpus were sharing a single hostcpu this could result in 'N - 1' unnecessary TLB invalidations. Since an 'invept' invalidates mappings for all VPIDs the first 'invept' is sufficient.
Fix this by moving the 'eptgen[MAXCPU]' array from 'vmxctx' to 'struct vmx'.
If it is known that an 'invept' is going to be done before entering the guest then it is safe to skip the 'invvpid'. The stat VPU_INVVPID_SAVED counts the number of 'invvpid' invalidations that were avoided because they were subsumed by an 'invept'.
Discussed with: grehan
|
#
261170 |
|
25-Jan-2014 |
neel |
Support level triggered interrupts with VT-x virtual interrupt delivery.
The VMCS field EOI_bitmap[] is an array of 256 bits - one for each vector. If a bit is set to '1' in the EOI_bitmap[] then the processor will trigger an EOI-induced VM-exit when it is doing EOI virtualization.
The EOI-induced VM-exit results in the EOI being forwarded to the vioapic so that level triggered interrupts can be properly handled.
Tested by: Anish Gupta (akgupt3@gmail.com)
|
#
261074 |
|
23-Jan-2014 |
neel |
Set "Interrupt Window Exiting" in the case where there is a vector to be injected into the vcpu but the VM-entry interruption information field already has the valid bit set.
Pointed out by: David Reed (david.reed@tidalscale.com)
|
#
261001 |
|
22-Jan-2014 |
neel |
Handle a VM-exit due to a NMI properly by vectoring to the host's NMI handler via a software interrupt.
This is safe to do because the logical processor is already cognizant of the NMI and further NMIs are blocked until the host's NMI handler executes "iret".
|
#
260863 |
|
18-Jan-2014 |
neel |
Some processor's don't allow NMI injection if the STI_BLOCKING bit is set in the Guest Interruptibility-state field. However, there isn't any way to figure out which processors have this requirement.
So, inject a pending NMI only if NMI_BLOCKING, MOVSS_BLOCKING, STI_BLOCKING are all clear. If any of these bits are set then enable "NMI window exiting" and inject the NMI in the VM-exit handler.
|
#
260836 |
|
18-Jan-2014 |
neel |
If the guest exits due to a fault while it is executing IRET then restore the state of "Virtual NMI blocking" in the guest's interruptibility-state field before resuming the guest.
|
#
260802 |
|
17-Jan-2014 |
neel |
If a VM-exit happens during an NMI injection then clear the "NMI Blocking" bit in the Guest Interruptibility-state VMCS field.
If we fail to do this then a subsequent VM-entry will fail because it is an error to inject an NMI into the guest while "NMI Blocking" is turned on. This is described in "Checks on Guest Non-Register State" in the Intel SDM.
Submitted by: David Reed (david.reed@tidalscale.com)
|
#
260619 |
|
13-Jan-2014 |
neel |
Add an API to rendezvous all active vcpus in a virtual machine. The rendezvous can be initiated in the context of a vcpu thread or from the bhyve(8) control process.
The first use of this functionality is to update the vlapic trigger-mode register when the IOAPIC pin configuration is changed.
Prior to this change we would update the TMR in the virtual-APIC page at the time of interrupt delivery. But this doesn't work with Posted Interrupts because there is no way to program the EOI_exit_bitmap[] in the VMCS of the target at the time of interrupt delivery.
Discussed with: grehan@
|
#
260532 |
|
11-Jan-2014 |
neel |
Enable "Posted Interrupt Processing" if supported by the CPU. This lets us inject interrupts into the guest without causing a VM-exit.
This feature can be disabled by setting the tunable "hw.vmm.vmx.use_apic_pir" to "0".
The following sysctls provide information about this feature: - hw.vmm.vmx.posted_interrupts (0 if disabled, 1 if enabled) - hw.vmm.vmx.posted_interrupt_vector (vector number used for vcpu notification)
Tested on a Intel Xeon E5-2620v2 courtesy of Allan Jude at ScaleEngine.
|
#
260531 |
|
11-Jan-2014 |
neel |
Enable the "Acknowledge Interrupt on VM exit" VM-exit control.
This control is needed to enable "Posted Interrupts" and is present in all the Intel VT-x implementations supported by bhyve so enable it as the default.
With this VM-exit control enabled the processor will acknowledge the APIC and store the vector number in the "VM-Exit Interruption Information" field. We now call the interrupt handler "by hand" through the IDT entry associated with the vector.
|
#
260466 |
|
09-Jan-2014 |
neel |
Don't expose 'vmm_ipinum' as a global.
|
#
260410 |
|
07-Jan-2014 |
neel |
Use the 'Virtual Interrupt Delivery' feature of Intel VT-x if supported by hardware. It is possible to turn this feature off and fall back to software emulation of the APIC by setting the tunable hw.vmm.vmx.use_apic_vid to 0.
We now start handling two new types of VM-exits:
APIC-access: This is a fault-like VM-exit and is triggered when the APIC register access is not accelerated (e.g. apic timer CCR). In response to this we do emulate the instruction that triggered the APIC-access exit.
APIC-write: This is a trap-like VM-exit which does not require any instruction emulation but it does require the hypervisor to emulate the access to the specified register (e.g. icrlo register).
Introduce 'vlapic_ops' which are function pointers to vector the various vlapic operations into processor-dependent code. The 'Virtual Interrupt Delivery' feature installs 'ops' for setting the IRR bits in the virtual APIC page and to return whether any interrupts are pending for this vcpu.
Tested on an "Intel Xeon E5-2620 v2" courtesy of Allan Jude at ScaleEngine.
|
#
260397 |
|
07-Jan-2014 |
neel |
Fix a bug introduced in r260167 related to VM-exit tracing.
Keep a copy of the 'rip' and the 'exit_reason' and use that when calling vmx_exit_trace(). This is because both the 'rip' and 'exit_reason' can be changed by 'vmx_exit_process()' and can lead to very misleading traces.
|
#
260383 |
|
06-Jan-2014 |
neel |
Allow vlapic_set_intr_ready() to return a value that indicates whether or not the vcpu should be kicked to process a pending interrupt. This will be useful in the implementation of the Posted Interrupt APICv feature.
Change the return value of 'vlapic_pending_intr()' to indicate whether or not an interrupt is available to be delivered to the vcpu depending on the value of the PPR.
Add KTR tracepoints to debug guest IPI delivery.
|
#
260380 |
|
06-Jan-2014 |
neel |
Split the VMCS setup between 'vmcs_init()' that does initialization and 'vmx_vminit()' that does customization.
This makes it easier to turn on optional features (e.g. APICv) without having to keep adding new parameters to 'vmcs_set_defaults()'.
Reviewed by: grehan@
|
#
260167 |
|
01-Jan-2014 |
neel |
Restructure the VMX code to enter and exit the guest. In large part this change hides the setjmp/longjmp semantics of VM enter/exit. vmx_enter_guest() is used to enter guest context and vmx_exit_guest() is used to transition back into host context.
Fix a longstanding race where a vcpu interrupt notification might be ignored if it happens after vmx_inject_interrupts() but before host interrupts are disabled in vmx_resume/vmx_launch. We now called vmx_inject_interrupts() with host interrupts disabled to prevent this.
Suggested by: grehan@
|
#
259942 |
|
27-Dec-2013 |
dim |
In sys/amd64/vmm/intel/vmx.c, silence a (incorrect) gcc warning about regval possibly being used uninitialized.
Reviewed by: neel
|
#
259863 |
|
25-Dec-2013 |
neel |
vlapic code restructuring to make it easy to support hardware-assist for APIC emulation.
The vlapic initialization and cleanup is done via processor specific vmm_ops. This will allow the VT-x/SVM modules to layer any hardware-assist for APIC emulation or virtual interrupt delivery on top of the vlapic device model.
Add a parameter to 'vcpu_notify_event()' to distinguish between vlapic interrupts versus other events (e.g. NMI). This provides an opportunity to use hardware-assists like Posted Interrupts (VT-x) or doorbell MSR (SVM) to deliver an interrupt to a guest without causing a VM-exit.
Get rid of lapic_pending_intr() and lapic_intr_accepted() and use the vlapic_xxx() counterparts directly.
Associate an 'Apic Page' with each vcpu and reference it from the 'vlapic'. The 'Apic Page' is intended to be referenced from the Intel VMCS as the 'virtual APIC page' or from the AMD VMCB as the 'vAPIC backing page'.
|
#
259782 |
|
23-Dec-2013 |
jhb |
Add a resume hook for bhyve that runs a function on all CPUs during resume. For Intel CPUs, invoke vmxon for CPUs that were in VMX mode at the time of suspend.
Reviewed by: neel
|
#
259542 |
|
18-Dec-2013 |
neel |
Use vmcs_read() and vmcs_write() in preference to vmread() and vmwrite() respectively. The vmcs_xxx() functions provide inline error checking of all accesses to the VMCS.
|
#
259205 |
|
10-Dec-2013 |
neel |
Fix x2apic support in bhyve.
When the guest is bringing up the APs in the x2APIC mode a write to the ICR register will now trigger a return to userspace with an exitcode of VM_EXITCODE_SPINUP_AP. This gets SMP guests working again with x2APIC.
Change the vlapic timer lock to be a spinlock because the vlapic can be accessed from within a critical section (vm run loop) when guest is using x2apic mode.
Reviewed by: grehan@
|
#
259085 |
|
07-Dec-2013 |
neel |
Use callout(9) to drive the vlapic timer instead of clocking it on each VM exit.
This decouples the guest's 'hz' from the host's 'hz' setting. For e.g. it is now possible to have a guest run at 'hz=1000' while the host is at 'hz=100'.
Discussed with: grehan@ Tested by: Tycho Nightingale (tycho.nightingale@pluribusnetworks.com)
|
#
259081 |
|
07-Dec-2013 |
neel |
If a vcpu disables its local apic and then executes a 'HLT' then spin down the vcpu and destroy its thread context. Also modify the 'HLT' processing to ignore pending interrupts in the IRR if interrupts have been disabled by the guest. The interrupt cannot be injected into the guest in any case so resuming it is futile.
With this change "halt" from a Linux guest works correctly.
Reviewed by: grehan@ Tested by: Tycho Nightingale (tycho.nightingale@pluribusnetworks.com)
|
#
258860 |
|
02-Dec-2013 |
neel |
The 'protection' field in the VM exit collateral for the PAGING exit is not used - get rid of it.
|
#
257422 |
|
31-Oct-2013 |
neel |
Rename the VMM_CTRx() family of macros to VCPU_CTRx() to highlight that these tracepoints are vcpu-specific.
Add support for tracepoints that are global to the virtual machine - these tracepoints are called VM_CTRx().
|
#
257297 |
|
29-Oct-2013 |
neel |
Remove unnecessary includes of <machine/pmap.h>
Requested by: alc@
|
#
256645 |
|
16-Oct-2013 |
neel |
Add a new capability, VM_CAP_ENABLE_INVPCID, that can be enabled to expose 'invpcid' instruction to the guest. Currently bhyve will try to enable this capability unconditionally if it is available.
Consolidate code in bhyve to set the capabilities so it is no longer duplicated in BSP and AP bringup.
Add a sysctl 'vm.pmap.invpcid_works' to display whether the 'invpcid' instruction is available.
Reviewed by: grehan MFC after: 3 days
|
#
256072 |
|
05-Oct-2013 |
neel |
Merge projects/bhyve_npt_pmap into head.
Make the amd64/pmap code aware of nested page table mappings used by bhyve guests. This allows bhyve to associate each guest with its own vmspace and deal with nested page faults in the context of that vmspace. This also enables features like accessed/dirty bit tracking, swapping to disk and transparent superpage promotions of guest memory.
Guest vmspace: Each bhyve guest has a unique vmspace to represent the physical memory allocated to the guest. Each memory segment allocated by the guest is mapped into the guest's address space via the 'vmspace->vm_map' and is backed by an object of type OBJT_DEFAULT.
pmap types: The amd64/pmap now understands two types of pmaps: PT_X86 and PT_EPT.
The PT_X86 pmap type is used by the vmspace associated with the host kernel as well as user processes executing on the host. The PT_EPT pmap is used by the vmspace associated with a bhyve guest.
Page Table Entries: The EPT page table entries as mostly similar in functionality to regular page table entries although there are some differences in terms of what bits are used to express that functionality. For e.g. the dirty bit is represented by bit 9 in the nested PTE as opposed to bit 6 in the regular x86 PTE. Therefore the bitmask representing the dirty bit is now computed at runtime based on the type of the pmap. Thus PG_M that was previously a macro now becomes a local variable that is initialized at runtime using 'pmap_modified_bit(pmap)'.
An additional wrinkle associated with EPT mappings is that older Intel processors don't have hardware support for tracking accessed/dirty bits in the PTE. This means that the amd64/pmap code needs to emulate these bits to provide proper accounting to the VM subsystem. This is achieved by using the following mapping for EPT entries that need emulation of A/D bits: Bit Position Interpreted By PG_V 52 software (accessed bit emulation handler) PG_RW 53 software (dirty bit emulation handler) PG_A 0 hardware (aka EPT_PG_RD) PG_M 1 hardware (aka EPT_PG_WR)
The idea to use the mapping listed above for A/D bit emulation came from Alan Cox (alc@).
The final difference with respect to x86 PTEs is that some EPT implementations do not support superpage mappings. This is recorded in the 'pm_flags' field of the pmap.
TLB invalidation: The amd64/pmap code has a number of ways to do invalidation of mappings that may be cached in the TLB: single page, multiple pages in a range or the entire TLB. All of these funnel into a single EPT invalidation routine called 'pmap_invalidate_ept()'. This routine bumps up the EPT generation number and sends an IPI to the host cpus that are executing the guest's vcpus. On a subsequent entry into the guest it will detect that the EPT has changed and invalidate the mappings from the TLB.
Guest memory access: Since the guest memory is no longer wired we need to hold the host physical page that backs the guest physical page before we can access it. The helper functions 'vm_gpa_hold()/vm_gpa_release()' are available for this purpose.
PCI passthru: Guest's with PCI passthru devices will wire the entire guest physical address space. The MMIO BAR associated with the passthru device is backed by a vm_object of type OBJT_SG. An IOMMU domain is created only for guest's that have one or more PCI passthru devices attached to them.
Limitations: There isn't a way to map a guest physical page without execute permissions. This is because the amd64/pmap code interprets the guest physical mappings as user mappings since they are numerically below VM_MAXUSER_ADDRESS. Since PG_U shares the same bit position as EPT_PG_EXECUTE all guest mappings become automatically executable.
Thanks to Alan Cox and Konstantin Belousov for their rigorous code reviews as well as their support and encouragement.
Thanks for John Baldwin for reviewing the use of OBJT_SG as the backing object for pci passthru mmio regions.
Special thanks to Peter Holm for testing the patch on short notice.
Approved by: re Discussed with: grehan Reviewed by: alc, kib Tested by: pho
|
#
255343 |
|
07-Sep-2013 |
neel |
Allocate VPIDs by using the unit number allocator to keep do the bookkeeping.
Also deal with VPID exhaustion by allocating out of a reserved range as the last resort.
|
#
253909 |
|
03-Aug-2013 |
grehan |
Follow-up commit to fix CR0 issues. Maintain architectural state on CR vmexits by guaranteeing that EFER, CR0 and the VMCS entry controls are all in sync when transitioning to IA-32e mode.
Submitted by: Tycho Nightingale (tycho.nightingale <at> plurisbusnetworks.com)
|
#
253849 |
|
31-Jul-2013 |
grehan |
Correctly maintain the CR0/CR4 shadow registers. This was exposed with AP spinup of Linux, and booting OpenBSD, where the CR0 register is unconditionally written to prior to the longjump to enter protected mode. The CR-vmexit handling was not updating CPU state which resulted in a vmentry failure with invalid guest state.
A follow-on submit will fix the CPU state issue, but this fix prevents the CR-vmexit prior to entering protected mode by properly initializing and maintaining CR* state.
Reviewed by: neel Reported by: Gopakumar.T @ netapp
|
#
249879 |
|
25-Apr-2013 |
grehan |
Add RIP-relative addressing to the instruction decoder. Rework the guest register fetch code to allow the RIP to be extracted from the VMCS while the kernel decoder is functioning.
Hit by the OpenBSD local-apic code.
Submitted by: neel Reviewed by: grehan Obtained from: NetApp
|
#
249450 |
|
13-Apr-2013 |
neel |
Create sysctl node 'hw.vmm.vmx' and populate it with oids that expose the VMX hardware capabilities.
Obtained from: NetApp
|
#
249351 |
|
11-Apr-2013 |
neel |
Make the code to check if VMX is enabled more readable by using macros instead of magic numbers.
Discussed with: Chris Torek
|
#
248935 |
|
30-Mar-2013 |
neel |
Add some more stats to keep track of all the reasons that a vcpu is exiting.
|
#
248389 |
|
16-Mar-2013 |
neel |
Allow vmm stats to be specific to the underlying hardware assist technology. This can be done by using the new macros VMM_STAT_INTEL() and VMM_STAT_AMD(). Statistic counters that are common across the two are defined using VMM_STAT().
Suggested by: Anish Gupta Discussed with: grehan Obtained from: NetApp
|
#
245917 |
|
25-Jan-2013 |
grehan |
Always allow access to the sysenter cs/esp/eip MSRs since they are automatically saved and restored in the VMCS.
Reviewed by: neel Obtained from: NetApp
|
#
245678 |
|
20-Jan-2013 |
neel |
Add svn properties to the recently merged bhyve source files.
The pre-commit hook will not allow any commits without the svn:keywords property in head.
|
#
245652 |
|
19-Jan-2013 |
neel |
Merge projects/bhyve to head.
'bhyve' was developed by grehan@ and myself at NetApp (thanks!).
Special thanks to Peter Snyder, Joe Caradonna and Michael Dexter for their support and encouragement.
Obtained from: NetApp
|
#
243667 |
|
29-Nov-2012 |
grehan |
Add support for the 0x81 AND instruction, now generated by clang in the local APIC code.
0x81 is a read-modify-write instruction - the EPT check that only allowed read or write and not both has been relaxed to allow read and write.
Reviewed by: neel Obtained from: NetApp
|
#
243651 |
|
28-Nov-2012 |
neel |
Cleanup the user-space paging exit handler now that the unified instruction emulation is in place.
Obtained from: NetApp
|
#
243650 |
|
28-Nov-2012 |
neel |
Change emulate_rdmsr() and emulate_wrmsr() to return 0 on sucess and errno on failure. The conversion from the return value to HANDLED or UNHANDLED can be done locally in vmx_exit_process().
Obtained from: NetApp
|
#
243640 |
|
27-Nov-2012 |
neel |
Revamp the x86 instruction emulation in bhyve.
On a nested page table fault the hypervisor will: - fetch the instruction using the guest %rip and %cr3 - decode the instruction in 'struct vie' - emulate the instruction in host kernel context for local apic accesses - any other type of mmio access is punted up to user-space (e.g. ioapic)
The decoded instruction is passed as collateral to the user-space process that is handling the PAGING exit.
The emulation code is fleshed out to include more addressing modes (e.g. SIB) and more types of operands (e.g. imm8). The source code is unified into a single file (vmm_instruction_emul.c) that is compiled into vmm.ko as well as /usr/sbin/bhyve.
Reviewed by: grehan Obtained from: NetApp
|
#
242331 |
|
29-Oct-2012 |
neel |
Convert VMCS_ENTRY_INTR_INFO field into a vmcs identifier before passing it to vmcs_getreg(). Without this conversion vmcs_getreg() will return EINVAL.
In particular this prevented injection of the breakpoint exception into the guest via the "-B" option to /usr/sbin/bhyve which is hugely useful when debugging guest hangs.
This was broken in r241921.
Pointy hat: me Obtained from: NetApp
|
#
242275 |
|
29-Oct-2012 |
neel |
Corral all the host state associated with the virtual machine into its own file.
This state is independent of the type of hardware assist used so there is really no need for it to be in Intel-specific code.
Obtained from: NetApp
|
#
242065 |
|
25-Oct-2012 |
neel |
If the guest vcpu wants to idle then use that opportunity to relinquish the host cpu to the scheduler until the guest is ready to run again.
This implies that the host cpu utilization will now closely mirror the actual load imposed by the guest vcpu.
Also, the vcpu mutex now needs to be of type MTX_SPIN since we need to acquire it inside a critical section.
Obtained from: NetApp
|
#
241982 |
|
24-Oct-2012 |
neel |
Maintain state regarding NMI delivery to guest vcpu in VT-x independent manner. Also add a stats counter to count the number of NMIs delivered per vcpu.
Obtained from: NetApp
|
#
241921 |
|
23-Oct-2012 |
neel |
Test for AST pending with interrupts disabled right before entering the guest.
If an IPI was delivered to this cpu before interrupts were disabled then return right away via vmx_setjmp() with a return value of VMX_RETURN_AST.
Obtained from: NetApp
|
#
241497 |
|
12-Oct-2012 |
grehan |
Add the guest physical address and r/w/x bits to the paging exit in preparation for a rework of bhyve MMIO handling.
Reviewed by: neel Obtained from: NetApp
|
#
241489 |
|
12-Oct-2012 |
neel |
Provide per-vcpu locks instead of relying on a single big lock.
This also gets rid of all the witness.watch warnings related to calling malloc(M_WAITOK) while holding a mutex.
Reviewed by: grehan
|
#
241147 |
|
02-Oct-2012 |
neel |
Get rid of assumptions in the hypervisor that the host physical memory associated with guest physical memory is contiguous.
Rewrite vm_gpa2hpa() to get the GPA to HPA mapping by querying the nested page tables.
|
#
240978 |
|
26-Sep-2012 |
neel |
Intel VT-x provides the length of the instruction at the time of the nested page table fault. Use this when fetching the instruction bytes from the guest memory.
Also modify the lapic_mmio() API so that a decoded instruction is fed into it instead of having it fetch the instruction bytes from the guest. This is useful for hardware assists like SVM that provide the faulting instruction as part of the vmexit.
|
#
240941 |
|
25-Sep-2012 |
neel |
Add support for trapping MMIO writes to local apic registers and emulating them.
The default behavior is still to present the local apic to the guest in the x2apic mode.
|
#
240912 |
|
25-Sep-2012 |
neel |
Add an explicit exit code 'SPINUP_AP' to tell the controlling process that an AP needs to be activated by spinning up an execution context for it.
The local apic emulation is now completely done in the hypervisor and it will detect writes to the ICR_LO register that try to bring up the AP. In response to such writes it will return to userspace with an exit code of SPINUP_AP.
Reviewed by: grehan
|
#
240894 |
|
24-Sep-2012 |
neel |
Stash the 'vm_exit' information in each 'struct vcpu'.
There is no functional change at this time but this paves the way for vm exit handler functions to easily modify the exit reason going forward.
|
#
239024 |
|
04-Aug-2012 |
neel |
Force certain bits in %cr4 to be hard-wired to '1' or '0' from a guest's perspective. If we don't do this some guest OSes (e.g. Linux) will reset the CR4_VMXE bit in %cr4 with disastrous consequences.
Reported by: grehan
|
#
238758 |
|
24-Jul-2012 |
neel |
Verify that VMX operation has been enabled by BIOS before executing the VMXON instruction.
Reported by "s vas" on freebsd-virtualization@
|
#
234761 |
|
28-Apr-2012 |
grehan |
MSI-x interrupt support for PCI pass-thru devices.
Includes instruction emulation for memory r/w access. This opens the door for io-apic, local apic, hpet timer, and legacy device emulation.
Submitted by: ryan dot berryhill at sandvine dot com Reviewed by: grehan Obtained from: Sandvine
|
#
228870 |
|
24-Dec-2011 |
grehan |
Add support for running as a nested hypervisor under VMWare Fusion, on systems with VT-x/EPT (e.g. Sandybridge Macbooks). This will most likely work on VMWare Workstation8/Player4 as well. See the VMWare app note at:
http://communities.vmware.com/docs/DOC-8970
Fusion doesn't propagate the PAT MSR auto save-restore entry/exit control bits. Deal with this by noting that fact and setting up the PAT MSR to essentially be a no-op - it is init'd to power-on default, and a software shadow copy maintained.
Since it is treated as a no-op, o/s settings are essentially ignored. This may not give correct results, but since the hypervisor is running nested, a number of bets are already off.
On a quad-core/HT-enabled 'MacBook8,2', nested VMs with 1/2/4 vCPUs were fired up. The more nested vCPUs the worse the performance, unless the VMs were started up in multiplexed mode where things worked perfectly up to the limit of 8 vCPUs.
Reviewed by: neel
|
#
222610 |
|
02-Jun-2011 |
jhb |
Some tweaks to the CPUID support: - Don't always pass the cpuid request to the current CPU as some nodes we will emulate purely in software. - Pass in the APIC ID of the virtual CPU so we can return the proper APIC ID. - Always report a completely flat topology with no SMT or multicore. - Report the CPUID2_HV feature and implement support for the 0x40000000 CPUID level. - Use existing constants from <machine/specialreg.h> when possible and use cpu_feature2 when checking for VMX support.
|
#
222605 |
|
02-Jun-2011 |
jhb |
Add a 'show vmcs' DDB command to dump state about the current CPU's current VMCS.
|
#
222112 |
|
20-May-2011 |
neel |
Fix a long standing bug in VMXCTX_GUEST_RESTORE().
There was an assumption by the "callers" of this macro that on "return" the %rsp will be pointing to the 'vmxctx'. The macro was not doing this and thus when trying to restore host state on an error from "vmlaunch" or "vmresume" we were treating the memory locations on the host stack as 'struct vmxctx'. This led to all sorts of weird bugs like double faults or invalid instruction faults.
This bug is exposed by the -O2 option used to compile the kernel module. With the -O2 flag the compiler will optimize the following piece of code:
int loopstart = 1; ... if (loopstart) { loopstart = 0; vmx_launch(); } else vmx_resume();
into this:
vmx_launch();
Since vmx_launch() and vmx_resume() are declared to be __dead2 functions the compiler is free to do this. The compiler has no way to know that the functions return indirectly through vmx_setjmp(). This optimization in turn leads us to trigger the bug in VMXCTX_GUEST_RESTORE().
With this change we can boot a 8.1 guest on a 9.0 host.
Reported by: jhb@
|
#
221914 |
|
14-May-2011 |
jhb |
First cut at porting the kernel portions of 221828 and 221905 from the BHyVe reference branch to HEAD.
|
#
221828 |
|
13-May-2011 |
grehan |
Import of bhyve hypervisor and utilities, part 1. vmm.ko - kernel module for VT-x, VT-d and hypervisor control bhyve - user-space sequencer and i/o emulation vmmctl - dump of hypervisor register state libvmm - front-end to vmm.ko chardev interface
bhyve was designed and implemented by Neel Natu.
Thanks to the following folk from NetApp who helped to make this available: Joe CaraDonna Peter Snyder Jeff Heller Sandeep Mann Steve Miller Brian Pawlowski
|