[hypervisor] Add exit address to VCPU exit trace

TEST=traced a debian_guest

Change-Id: Ie24e0943e2734b54b48f35dd842420923c81899f

[mmu][hypervisor] Support for more memory types

Add support for uncached, uncached device, and write-combining to
stage-2 page tables on arm64 and EPT page tables on x86-64.

Test: Ran 'guest launch zircon_guest', 'hypervisor-test',
'machina_unittests', and 'k ut hypervisor'.
Change-Id: I9d8e55c8357384e04a099c1cb25ac977027ed239

[hypervisor] Make zx_guest_create return a VMAR

Modify zx_guest_create to return a VMAR that represents the guest
physical address space.

A follow up CL will cover changes to the MMU code that allows uncached
and write-combining VMOs to be mapped into the guest physical address
space VMAR.

Test: Ran 'vmar-test', 'hypervisor-test', 'machina_unittests',
and 'guest launch zircon_guest'
Change-Id: I4b9eada3ccb206a205c6b30f1cf882766ba70fed

[ulib][zx] Add VCPU syscall wrapper

Also update the hypervisor tests to use the syscall wrappers.

PD-167 #done

Test: Passed CQ.
Change-Id: Iaed8b743461c981b1d71090ea44e488bc69a1e68

[x86][hypervisor][vm] Avoid lgdt/lidt instructions on every VM exit

On VM exit CPU implictly sets GDT limit to 0xffff even if before
VM entry it was different. So we have to call LGDT on VM exit to
restore the limit.

However GDT limit is just a safe guard that causes GP fauilure if
someone tries to load a segment register with a value pointing
outside GDT. It's not accurate though, for example, space allocated
for GDT and the value of GDT limit depends on the value of
SMP_MAX_CPUS constant, and if SMP_MAX_CPUS doesn't match perfectly
the actual number of cpus, we end up allocating more memory than
needed.

So instead of caring about preserving GDTR limit let's just reserve
0x10000 bytes and map just enough pages to store the actual GDT and
leave the rest unmapped. One benefit of this is that LGDT is a
serializing instruction according to Intel SDM.

Note that unlike IDT we are currently modify GDT in the hypervisor
code after VM exit to restore TSS limit, so we use rw mapping around.

With IDT it's simpler, because it doesn't seem that IDT limit
affects anything unless it's smaller than 256 * 16 - 1, so we can
just get rid of lidt altogether.

Test: manually on Intel NUC and QEMU/KVM, k ut all, runtests

Change-Id: I00f87c24621e38d688859e7bbcea775a5b3f0f80

[x86][hypervisor] Cancel lapic timer on Vcpu destruction

This prevents a panic when the timer fires after the object has been
deleted.

TEST=guest_integration_tests (in development)

Change-Id: I1fdbdb4f2568f1bfcb55afd181b2a06cdec6bf77

[hypervisor][x86] Remove sysenter MSRs from guest and host msr pages

These MSRs are stored in the guest and host state so they're not needed
here.

Also, ignore upper 32 bits of rcx for msr handling.

Change-Id: Ia70980dbed920e71dffb75badfc80fecb7f9b58d

[kernel][mp] add new mp_interrupt() call to be used by the hypervisor

In some cases the hypervisor just needs to interrupt another cpu for the
purposes of getting it to vmexit. Add another specific mp_* call to just
do this. Allows for simplifying the mp_reschedule() call by removing a
now unused flag.
Remove a layer of indirection of the ipi interrupts which aren't useful.

Change-Id: Ib54fc8683338b6f9a8efeeb0e7478a0a5cdab095

[x86][hypervisor] Allow guests to use sysenter MSRs

Linux guests will use sysenter from compatibility mode regardless of the
SEP bit, so we must support these MSRs.

Change-Id: I59eb9eca2e68a2867820c1263681f84e6c4c812e

[syscalls] Update parameter types and names

This CL updates the parameter types and names for a number of syscalls
to match the Zircon System API rubric. There's more work to do to make
all the syscalls conform to the rubric, but this change is a start.

Change-Id: I218ac5e7e0cbd80a8c69fef7891ad40b78b6b407

[x86][hypervisor] Move default operand size to guest mem packet

Change-Id: I25bbba02704beb4f657acc59c29e9d6e7296f582

[x86][registers] Add API to expose xsave info.

The format of the xsave legacy area is moved to the header and all
values are filled out (previously this was just in registers.cpp and
only mxcsr was defined).

The existing #defines for states were renamed *_BIT_* since they're bit
masks, and a new set of defines for the state indices was added. These
are used for referring to the state number in the new API.

Logic for rounding the xsave area was moved to
x86_extended_register_init_state from thread.cpp. This isn't strictly
necessary (originally this was to support a version of this patch that
was removed) but this seems like a cleaner split of responsibilities. In
support of this, a new define for the required buffer size is added.

Change-Id: I91c4ee2d27419f3b6bb6f23582ea7275a3566396

[x86][hypervisor] Add the VCPU's default operand size to vcpu_state

We need this for instruction decoding.

Change-Id: Ie0b313ef8b12f7ad9c8726b5b2daf4393c34f4aa

[x86][hypervisor] Check clocksource is stable once

When we first create a VCPU, check whether the clocksource is stable.
This saves us from checking each time we update the system time.

Change-Id: I3eed90ea22ad0237abdea54220953c8206761499

[hypervisor][ktrace] Add VCPU blocking events

Add ktrace events for when the VCPU is blocked while either waiting for
an interrupt or waiting for a port packet to become available.

This also restructures the way we record metadata for VCPU ktraces in
order to be more efficient.

PD-100 #comment

Change-Id: I43551a0068bb3d6393a9a58c21b3ea813edfb726

[hypervisor] Add ktrace to VCPU

Add tracing of VCPU enter and exit, as well as VM exit reasons.

Change-Id: Iad8c544219f8e2a5da8fd33a0868717a2634879d

[x86][hypervisor] Reinstate EPT invalidation

We used to invalidate EPT caches when launching a guest due to the use
of APIC addresses, which was removed when we moved to X2APIC. However,
we still need to invalidate the cache to cover the case where a guest is
destroyed immediately before a new guest with the same EPTP is started.
This was occurring randomly when running hypervisor-test in a loop and
causing triple faults.

PD-73 #done

Change-Id: Id250f52d7e3023daafcba78b0966667b5f9c78ad

[x86][hypervisor] Mask CR0.NE and handle writes to CR0

Linux writes values to CR0 that are incompatible with VMX on secondary
processor start-up. Here we mask CR0.NE, which is the only bit that we
need to keep set on secondary processors, to cause a VM exit on writes
to CR0 that don't set NE. The downside of this is that we need to
manually handle the CR0 write including entering long mode when the
guest sets CR0.PG and EFER.LME.

Change-Id: I3acc0c01b938de6f52e57f2fd4bc476c7b7ce185

[test][hypervisor] Fix interrupt tests

Use interrupt 32 as the test vector. Previously 0 was used, which is
counted as an NMI and injected into the guest before it has set up the
IDT.

Also, a slight change to interrupt injection code. Vectors 0-31 are
exceptions and so should by labeled as hardware exceptions when passed
to the guest. However, breakpoint and overflow exceptions should be
software exceptions. Previously all vectors up to the max Intel defined
vector were reported as hardware exceptions (and also considered to be NMIs),
here we lower that to only include vectors below the user defined base (32).

Change-Id: I067a7befac3a65b2aede692aefcfc79b74536de0

[kernel][sched][x86] Use monitor/mwait when idle

When rescheduling idle cpus, use monitor/mwait instead of relying on
IPIs. This change adds some reschedule-specific arch hooks, instead of
relying on the arch IPI hooks.

The x86 percpu state includes a variable that tracks whether the cpu is
running the idle thread. The idle thread monitors and mwaits on that
variable. Then other threads can reschedule the idle cpu by clearing
the monitored variable.

ZX-1713
ZX-1293 #done

Change-Id: I5f7bf073e3e5b6e1e5fa4febc412f52f40773e2d

[x86][hypervisor] Enable pause exiting

This will allow us to use a pause to exit the guest when waiting for the
pvclock to update.

Change-Id: I95761131e3d1d14f7df95ab04776fc3ba034492a

[x86][hypervisor] Disable HALT IPI short-circuiting.

This causes failures on x86 linux guests.

Change-Id: Ic98193d23d0048a3e98fede78786f50f2e2ebd6b

[x86][hypervisor] Short circuit HALT IPIs.

This will allow us to stop a VCPU regardless of guest HALT handling.

Change-Id: I1938bb43120ed88fb43c78e8765326dbb55cd8c2

[x86][hypervisor] Start secondary VCPUS in real mode.

Use the unrestricted guest extension to start secondary VCPUs without
protection or paging.

Change-Id: I13cae2ef6401455cb3d26773435d4d6e77290a44

[lib][hypervisor] Move everything into a namespace

Move everything, but vmm_guest_page_fault_handler, into the hypervisor
namespace.

Change-Id: Ied96a7945b26c4a1ed24c03b293d42c7d89e7812

[arm64][hypervisor] Allocate a page for El2State

When we pass El2State from the host to EL2, we go from the host virtual
address to the 1:1 physical address mapping in EL2. This means that
EL2State can cross a page boundary unless we're careful, which would
cause us to write into the wrong memory if we're not careful.

Change-Id: I2ed2c0b0fe9c3981ac920f87e5307508cf3331ff

[arm64][hypervisor] Invalidate cache on MMU enable

When the MMU is enabled, invalidate the guest physical address space.

Change-Id: I93d66ec51b5762e309f8fc8db462923a26590e09

[kernel][hypervisor] KVM clock support in hypervisor

KVM/Xen paravirtualized clock API consists of monotonic clock
source (we refer to it as system time) and fixed point in time
(we refer to it as boot time), and boot time + system time should
give current wall time. Since system time is supposed to be
monotonic it means that boot time may be adjusted (for example,
by NTP) and therefore we use sys_get_clock to get boot time.

System time is exported per VCPU and periodically updated by host.
TSC is used to adjust system time value in the guest between
those updates. We update system time on VCPU resume.

Getting system time in the guest is expected to be fast, but boot
time on the other hand is not expected to be fast and only
guaranteed to be updated by host after MSR write from guest and
therefore incurs vmexit. Despite the name boot time doesn't need
to be guest boot time. In KVM it's actually host boot time and we
use the same boot time for this implementation.

And one comment regarding naming. KVM and Xen use the same
structures layout for paravirtualized clock, but KVM uses MSRs
to communicate with guests and Xen uses PV ops for that. So data
strcutures are common and use prefix pv or Pv, while constants
describing MSRs and KVM specific flags use Kvm prefix.

Change-Id: I81cdbd893985ad17bcf5f163e652ec1bb63feacb

[hypervisor] Allocate VPIDs per guest instead of globally

ZX-1452 #done

Change-Id: I5274bf94d3d6807c582aab8c7ecef683730df0cb

[x86][hypervisor] Remove unused APIC page code

Now that we've moved to x2APIC, this code is no longer used.

Change-Id: If4ad714c46be78150a25e0b10671dfbda4d85551

[hypervisor][x86] Route faults through the InterruptTracker.

This ensures we only inject a single vector each vm enter.

ZX-1632 #done

Change-Id: Ibf6281b2d67f3598c9c04909272231e5c790eb4a

[arm64][hypervisor] Fix WFI/WFE handling

This moves the timer handling back to WFI/WFE events. Without this, we
were injecting far too many timer interrupts, causing constant world
switches between the guest and the host.

Change-Id: Ic78c8d92943d52c4fdd90e3568f108843d00aa78

[debug] Fix compiler nits so we can build with -Og

All changes were related to uninitialized variables accesses
that are optimized out in other builds.

Change-Id: Ide3d49fb7dcbf8211fa7a3624b7119e837c62019

[hypervisor] Replace zx_vcpu_create_args with entry.

The initial CR3 on BSPs is always 0 and APs begin without paging then
discover the CR3 on their own, so including this as an argument is
unnecessary.

(Also clang-fmt'd)

Change-Id: I88567732252538a0eb239ee1ef4167bf0570f4c1

[kernel][hypervisor] Remove some always-true "reschedule" args

Remove the "bool reschedule" argument from
hypervisor::InterruptTracker's Signal() and Interrupt() methods.

ZX-1490

Change-Id: Ibcbbc9b0c8b51978d29b073882727e37091e4011

[hypervisor] Track when the guest is running

This allows to invoke mp_reschedule in Vcpu::Interrupt only if the VCPU
is actually running. It also simplifies the logic of Vcpu::Interrupt,
requiring less reading of tea leaves.

Change-Id: I0fd363e39838a66d53447c342ae0d399315f843f

[hypervisor] Swizzle args for functions

Separating out this cosmetic change from an upcoming change.

Change-Id: I9b2caa49a39a23e260a7c2ded36a36dce517ebff

[arm64][hypervisor] Don't inject timer for WFI

When we exit for a WFI, we should wait until the timer expires and the
re-enter on the next instruction. But we should not inject a timer
interrupt.

This fixes issues with Linux guests on arm64 once they enter
arch_cpu_idle.

Change-Id: I67db6ce7fcc03800f55100f209a8b0ae85e4777a

[hypervisor][x86] Add X2APIC support and pretend to be KVM.

Linux doesn't support X2APIC without IRQ remapping unless it recognises
that it's running as a guest. For now we can pretend to be KVM to
trigger hypervisor detection.

ZX-1257 #done

Change-Id: I66094b65f4299c5a6a51eab8ce24f419e05d23f7

[kernel][hypervisor] Do not require INVPCID support

INVPCID is an optimization that allows to avoid flushing TLB, it
should be possible to run hypervisor without VMX support for INVPCID.
Particularly when using KVM in nVMX mode, nested VMX doesn't support
it, at least in not bleeding edge KVM versions.

This change makes VMX INVPCID support optional and clears INVPCID flag
in CPUID if VMX doesn't support it, thus proper guests shouldn't use it
in this case.

Change-Id: Ide1c2da57471f6c223c9888ecf7eacbd87a7aae8

[x86][hypervisor] Remove AutoVmcs::Reload logic

Every path that called Reload was immediately returning, to then have a
new instance of AutoVmcs reload the VMCS. The call to AutoVmcs::Reload
is redundant.

Instead, we replace AutoVmcs::Reload with AutoVmcs::Invalidate. All
paths that are blocking will require a StateInvalidator, which will then
invalidate the AutoVmcs so that any use of it after invalidation will
trip a DEBUG_ASSERT, making failure much more explicit than current.

Change-Id: I2497e63d7acc1de97571354ffdd05dc7e2c8cef5

[hypervisor] Refactor common interrupt logic

Introduce interrupt_tracker, so that we can share common interrupt
tracking logic between arm64 and x86.

Furthermore, this improves interrupt handling on arm64, bringing it up
to the same level as x86.

We also now inject the timer interrupt correctly on arm64.

Change-Id: I09c4d37667b9e7c14831400875bc972df64f97de

[x86][hypervisor] Cleanup constant declarations.

Stop using #defines, and prefer static const. MSR constants remain the
same due to matching with existing constants elsewhere. Also remove
redundant comments that add no information.

Change-Id: I2b8c7eb38d1fe429e3c38efd870966004c73cd57

[x86][hypervisor] Fix issuing multiple interrupts on one vmexit.

Pending interrupts are being issued immediately prior to resuming the
VCPU so these calls are causing interrupts to be dropped when multiple
interrupts are queued up.

Change-Id: I2e869876734105f1278601b3ea2b5fc39ec4f8dc

[kernel][memory] refactor some physical<->virtual routines, formally naming the BKM physmap

This mostly just rearranges the names of things, but add an actual name
for the big kernel map and add some bounds checking to usage.

This should let us be a bit more dynamic about the physmap in the
future, including placing it at a random location and/or giving it a
more complex mapping.

Change-Id: I062d3f0483f27436252b9607174e10b852f0f832

[x86][hypervisor] Inject interrupts on VCPU resume.

If there are interrupts pending, inject them at VCPU resume instead of
waiting for an EOI. This also corrects the situation where we are
executing on the same CPU as the VCPU when we issue an interrupt.
Without this, the interrupt processing would be delayed until an EOI was
seen.

ZX-1340 #done

Change-Id: I4dabbc616a8750190a424948982b0e1b0b911b5e

[hypervisor] Avoid calling free_vpid twice in the error path

Now when Vcpu takes ownership of the allocated vpid it's possible
that free_vpid may be called twice: once from the destructor and
another one from auto_call. Move auto_call.cancel() to avoid it.

Change-Id: I4091c2fb18cd69eea8ef4153bb0129abfce00a2a

[x86][hypervisor] Invalidate EPT before first use.

This ensures that if the EPTP coicidentally ends up being the same as a
previously used EPTP, we don't reuse any stale mappings.

Also remove MiscInfo, as it's not currently used for anything, but does
prevent the use of nested-VMX on KVM.

ZX-981 #comment

Change-Id: I9662d28b56a51742725f08ac8ac98db2441c0af7

[lib][hypervisor] Move common pinning functions.

We now pin the thread, and validate that it is pinned correctly,
uniformly across x86 and arm64.

This also fixes a VPID leak in the arm64 VCPU, and simplifies the
pinning functions.

Change-Id: I247f6d8d09cebfba4e7d2cc92428433b335e85ea

[lib][hypervisor] Rename CpuState to IdTracker.

The plan is to use IdTracker on ARM64 to track VMID and VPID separately,
as GICv2 will limit the number of VPIDs that can be active.

ZX-1289 #comment

Change-Id: I2d3a93eaa779778cd2e50a027f586a8fb2371d65

[x86][hypervisor] Cleanup some tiny nits.

We should settle on using single-line comments.

Change-Id: I971eef3ec51570496a88e92aed73d7413120a8c0

[hypervisor][x86] Enabled "ack-interrupts-on-vmexit"

With this feature enabled if vmexit happens due to external interrupt CPU
acknowledges the interrupt and provides us with vector number. We can
examine this information and call the handler directly.

ZX-1254 #done

Change-Id: I4472fa037c731c1943b7c035e471523a2b70e46f

[x86][hypervisor] Minor cleanups to x86 code.

Change-Id: I076c1bfc3df3dfcd623d68dc6ed2a7a4a3522fe5

[kernel][x86] Fix naming of X86_INT_MAX and add X86_INT_COUNT

Change-Id: I5a231b9c26cd4cc4336fb3d90c8cb30ba8465fcb

[lib][hypervisor] Rename PacketMux to TrapMap.

This more accurately describes what it does, and makes more sense in
the code.

Change-Id: Iae0223645dbf4b399ce29e15e10a8d3eeb8d3cf5

[kernel][sched] thread's cpu affinity is now a bitmask instead of a number

-Can specify the mask of which cpus a thread is allowed to run on
-thread_set_cpu_affinity() will now do what is needed to migrate a
thread between cores, even if the thread is currently running on another
cpu.

Change-Id: I5dfc9f8e1c23d63ed749c84bc3dfa4118470b8e6

[kernel][mp] add new header with types and routines to deal with cpu numbers

Add a few more types and switch some apis to using those. No functional
change.

Change-Id: I67add1247cf36d9e6a55f15dd809ffe4bafe06fd

[x86][hypervisor] Minor cleanup.

Be explicit in the lambda capture for the auto-call.

Change-Id: I8c3f09ab6e32156e1c26968fd75c54495d9334ff

[x86][hypervisor] Move pin_thread to cpu_state.h.

This is so that it can be shared between x86-64 and arm64.

Change-Id: I97b9cc9af4afa110a2107487c3061de22153d663

[x86][hypervisor] Make zx_vcpu_state match zx_x86_64_general_regs.

If ARM64 ends up matching in structure too, we can reuse the debug
structures.

Change-Id: I159381a015863025e15b761e1a162f55d0c0ef49

[zx] Magenta -> Zircon

The Great Renaming is here!

Change-Id: I3229bdeb2a3d0e40fb4db6fec8ca7d971fbffb94

[hypervisor] Extract common CPU state logic from VmxCpuState.

This is so we can reuse some of this logic in El2CpuState, and
customise it for allocating VMIDs.

Change-Id: I05b76d9694b8ed4446763604239cb78f77786e7f

[zircon][mxtl->fbl] Rename mxtl to fbl

Change-Id: Ie21b6498e1bfb0a7fa0315e40b9e5c3ee78646be

[kernel][vm] move kernel/kernel/vm to just kernel/vm

Change-Id: I8f724a9f8a61415712661d1fdd3dc4e1c70cf620

[hypervisor] Replace all status_t with mx_status_t.

As requested from a previous CL. (Also ran clang-format.)

Change-Id: Ib4106f3c03f2084755c596062d34c8f72e42dcdb

[kernel][mp] change the inter-processor-interrupt api to take a enum and mask

This removes the special casing of a few mask values to mean all-cpus or
all-but-local. Also means we can push the SMP_MAX_CPUS up to 32 before
switching to a different mask type.

Change-Id: I89276c2d0acf77ae2df132beb9ef1dcd5ef3b7e9

[hypervisor] Make guest traps use ports.

Ports are a much better fit for the hypervisor than FIFOs. To take
advantage of ports, we use our arena for PortPackets, rather than
relying on the system-wide arena. This also allows us to easily wait
and signal in response to packet availability in our own per-trap
arenas.

Change-Id: I8664f8438710e38abf96aefedc6ba446aaf04683

[x86][hypervisor] Run clang-format over code.

Run clang-format over x86 hypervisor code.

Change-Id: I8616a2e5c4b3865bf9fdcac5627c833bc21bb39c

[mxtl][mxalloc] Redirect all clients of mxalloc to mxtl.

Change-Id: Ib6d88937212ab23612f59add8bd028862308ead5

[x86][hypervisor] Return MX_ERR_BAD_STATE if VCPU invariant is violated.

Rather than using DEBUG_ASSERT, we should return an error.

Change-Id: I1fbadba32e52d681050d11260a7028ffc99c794a

[magenta] Add syscalls for the hypervisor.

This change adds 7 syscalls for the hypervisor:
* guest_create - create a guest
* guest_set_trap - sets a trap within a guest
* vcpu_create - create a VCPU
* vcpu_resume - resume execution of a VCPU
* vcpu_interrupt - issue an interrupt on a VCPU
* vcpu_read_state - read the state of a VCPU
* vcpu_write_state - write the state of a VCPU

This formalizes the hypervisor ops that were previously used by
promoting them to first-class syscalls.

Change-Id: I5d8e3772b98ae423faa39d21a6ef2901e940ffd9

[x86][hypervisor] Fix use of mp_reschedule.

The flags passed to mp_reschedule should just be 0.

Change-Id: I40eb6ca48546846c3eb6bc5121cad2797a480a4f

[hypervisor] Restructure around VCPUs.

This changes the structure of the hypervisor to be modelled around
virtual CPUs. Each VCPU is associated with a thread and a CPU, in
order to increase performance and simplify communication with user
space.

While the kernel-side of the hypervisor has been restructured, the
user space-side has not been modified to take advantage of the
optimisations that are possible. This will come in a follow up CL.

Change-Id: Ic0c9be724f12c9de601fc4b892d74ba224b0d85e