[hypervisor] Add exit address to VCPU exit trace

TEST=traced a debian_guest

Change-Id: Ie24e0943e2734b54b48f35dd842420923c81899f

[hypervisor] Set fault flags based on VMAR flags

When we fault in pages for the guest physical address space, use the
VMAR flags for the faulting address to apply the correct fault flags.

This allows users to map VMOs into the guest VMAR that have a more
limited set of permissions.

MAC-166 #done

Test: Ran "k ut hypervisor", "linux_guest", and "zircon_guest".
Change-Id: I9d03b0b609daf20355fdb552a49b789b63830084

[hypervisor] Separate WFI_WFE_INSTRUCTION tracing event

TEST=Traced runs of guest_integration_tests

Change-Id: I382ff71e9071de326a97e34eb9343faf8de89c4e

[hypervisor] Add better output when vmexit_handler fails

TEST=Run zircon_guest on x64 and arm64

Change-Id: I1b158d8c698608679ab88e0f197d93a32f7caf36

[hypervisor] Increase interrupt packet mask size to 64 bits

There are no changes to arm64 since we don't support affinity yet so
mask contains only the least significant 8 bits. See MAC-156.

TEST=Build and run zircon_guest and linux_guest

Change-Id: Id5d07ef4abcdb87d4aba249c170d02067f53d83f

[hypervisor] Make zx_guest_create return a VMAR

Modify zx_guest_create to return a VMAR that represents the guest
physical address space.

A follow up CL will cover changes to the MMU code that allows uncached
and write-combining VMOs to be mapped into the guest physical address
space VMAR.

Test: Ran 'vmar-test', 'hypervisor-test', 'machina_unittests',
and 'guest launch zircon_guest'
Change-Id: I4b9eada3ccb206a205c6b30f1cf882766ba70fed

[arm][hypervisor] Handle ICC_SGIR_EL2.

This register is used by the guest to signal a software generated
interrrupt when using GIC v3.

This change requires a follow up in the garnet layer to be useful for
zircon_guest, and more work to be useful for linux_guest.

TEST=In qemu: guest launch zircon_guest --display=none --gic=3.

Change-Id: I22636fa7d68130507aa96a33386f538415d903c5

[hypervisor][arm64] Trap guest debug reg access

To enable debugging on ARM64 we have to unlock
the OS Lock upon boot. We don't want the guest OS
to alter the state of the OS lock so we trap any
accesses to the debug registers and implement
them as WI/RAZ.

This should work as both Zircon and Linux unlock
the debug lock immediately once upon boot
regardless of whether or not the debug lock
is implemented.

An guest OS that attempts to query for the
presense of the debug lock will find that it is
unimplemented since the OSLSR is implemented as
RAZ.

Change-Id: Ifa42f4fa2a50fc3569eac9a194c19a33059d7151

[arm64][hypervisor] Clean as well as invalidate page table caches on enabling MMU

Change-Id: I1774487e39e5546e44c9ef2315f138605c61cf80

[arm64][hypervisor] Support PSCI CPU_ON calls

This allows us to bring up multiple VCPUs.

Also, use the CPU-agnostic address for GICH.

Change-Id: Iefff3c8fcdc0bb3d948dd0cb2b61c19dd9ed20ad

[hypervisor][ktrace] Add VCPU blocking events

Add ktrace events for when the VCPU is blocked while either waiting for
an interrupt or waiting for a port packet to become available.

This also restructures the way we record metadata for VCPU ktraces in
order to be more efficient.

PD-100 #comment

Change-Id: I43551a0068bb3d6393a9a58c21b3ea813edfb726

[hypervisor] Add ktrace to VCPU

Add tracing of VCPU enter and exit, as well as VM exit reasons.

Change-Id: Iad8c544219f8e2a5da8fd33a0868717a2634879d

[arm64][hypervisor] Prefer timer interrupts

When there are many interrupts pending from devices, there may not be
any spare list registers for the timer interrupt. If the timer interrupt
isn't fired on time, Linux's RCU handling becomes unhappy. Therefore
always give preference to timer interrupts.

PD-88 #done

Change-Id: I2bad326f82805b10a19a4208b6c7729f358ad3b4

[hypervisor] Enforce port-requirements for traps

This ensures that BELL traps always have a port associated with them,
and all other types do not.

Change-Id: Iaff6f8f49b3c342cc474afe8e783047d3e05b2d6

[lib][hypervisor] Move everything into a namespace

Move everything, but vmm_guest_page_fault_handler, into the hypervisor
namespace.

Change-Id: Ied96a7945b26c4a1ed24c03b293d42c7d89e7812

[arm64][hypervisor] Invalidate cache on MMU enable

When the MMU is enabled, invalidate the guest physical address space.

Change-Id: I93d66ec51b5762e309f8fc8db462923a26590e09

[arm64][hypervisor] Warn on unhandled sysreg

If we encounter an unhandled system register, we should complain loudly.

Change-Id: I8bc4ce529eb46c86e8556e0f6092b40edbaa4c16

[arm64][hypervisor] Fix WFI/WFE handling

This moves the timer handling back to WFI/WFE events. Without this, we
were injecting far too many timer interrupts, causing constant world
switches between the guest and the host.

Change-Id: Ic78c8d92943d52c4fdd90e3568f108843d00aa78

[arm64][hypervisor] Disable VM trap iff cache on

Disable traps to virtual memory control registers if and only if caching
is on. This is being paranoid, in practice this isn't required, but
better to be safe.

Change-Id: I02ee7d75d56a2becb309b87aa54bb6442d5ad1c3

[arm64][hypervisor] Disable VM traps if MMU is on

We only need to trap writes to virtual memory control registers while
the MMU is off. Once it is on, we can disable the traps to reduce the
number of guest exists and improve performance.

Change-Id: I47e3eabef314b9cfb1fd96f2104844dee09a83c6

[arm64][hypervisor] Change how we handle timers

On every VM exit, we check whether the guest timer was set. If so, we
setup a timer object to fire and inject an interrupt when the guest
expected one.

Previously, we would wait for a WFI before we setup a timer object,
which would cause us to incorrect inject a timer.

We also now ensure that interrupts are unmasked before we inject them.

Change-Id: I1a1a2319a6d358f5a86d45ee251bafb6156a22bd

[kernel][hypervisor] Convert timer callbacks to use preempt_disable

Calling sched_reschedule() when arch_in_int_handler() is true is now
allowed (see preempt_disable). This means we can remove uses of the
INT_RESCHEDULE return value. We can instead call sched_reschedule()
or pass reschedule=true to functions that take a "reschedule"
argument.

ZX-1490

Change-Id: I2f777296db65558c4124fc8a522932fcf174bf82

[hypervisor] Add additional logging for unmapped IO.

Change-Id: Idc1b9c3ee3f18890ef9a030b5bb2d1d819267d4c

[arm64][hypervisor] Don't inject timer for WFI

When we exit for a WFI, we should wait until the timer expires and the
re-enter on the next instruction. But we should not inject a timer
interrupt.

This fixes issues with Linux guests on arm64 once they enter
arch_cpu_idle.

Change-Id: I67db6ce7fcc03800f55100f209a8b0ae85e4777a

[arm64][hypervisor] Save guest virtual timer

Make sure we save the guest the virtual timer on VM exit. We then rely
on these saved values for timer handling, rather than on the current
value of the timer registers which may have been modified by the host.

Furthermore, this CL restructures the El2State and modifies the handling
of HostState in el2.S, so that we can increase the size of El2State
without hitting the limits of immediates in instructions.

Change-Id: I110397094be4f40bdce1ced25390554415c5d1c3

[kernel][arm64] clang format arm64 code

Change-Id: Id4b2123b16549c555db28bc878dde434f4dc5506

[arm64][hypervisor] Use virtual timer

Use virtual timer, instead of physical timer. Both Zircon and Linux are
configured to use it.

Change-Id: I0ad15c425932b86dd25be72ea3820623842db184

[x86][hypervisor] Remove AutoVmcs::Reload logic

Every path that called Reload was immediately returning, to then have a
new instance of AutoVmcs reload the VMCS. The call to AutoVmcs::Reload
is redundant.

Instead, we replace AutoVmcs::Reload with AutoVmcs::Invalidate. All
paths that are blocking will require a StateInvalidator, which will then
invalidate the AutoVmcs so that any use of it after invalidation will
trip a DEBUG_ASSERT, making failure much more explicit than current.

Change-Id: I2497e63d7acc1de97571354ffdd05dc7e2c8cef5

[hypervisor] Refactor common interrupt logic

Introduce interrupt_tracker, so that we can share common interrupt
tracking logic between arm64 and x86.

Furthermore, this improves interrupt handling on arm64, bringing it up
to the same level as x86.

We also now inject the timer interrupt correctly on arm64.

Change-Id: I09c4d37667b9e7c14831400875bc972df64f97de

[arm64][hypervisor] Set timer on WFI

If the timer is enabled, set a timer before sleeping on a WFI. Once the
timer expires, we exit from sleep and re-enter the guest.

ZX-1347 #done

Change-Id: Ia2ec38e772876e4e64c4865b7a1369d2444eba98

[arm64][hypervisor] Do not trap WFE

WFE is typically used for short waits, so trapping to the hypervisor
would greatly reduce performance. Furthermore, there doesn't seem to be
a way to trap an SEV to use as an event to re-enter the VCPU.

ZX-1347 #comment

Change-Id: I9bc973521d93ad75f3a99e71f3665ddeb3bb1c75

[arm64][hypervisor] Handle PSCI64_CPU_ON calls.

For now, we simply ignore attempts to start up other CPUs, and return an
error code to the guest.

ZX-1347 #comment

Change-Id: I99ad88e36c28115707a6680cc2bca08fbbe3d5f5

[arm][hypervisor] Add wfi support

Adds support for wfi (wait for interrupt) instruction to vmexit.

Change-Id: I9131e4d3527f57d00dcc9a9e69782755ea3d5c89

[arm][hypervisor] Use FAR_EL2 for mem trap.

Combine the lower bits of FAR_EL2 with the HPFAR_EL2 to get the exact
address that cause a mem trap.

Without this change, we only had page-level granularity for a mem trap.

ZX-1347 #comment

Change-Id: I19aef7bd84ba78f7b6d43be7374399b3cc1cbba2

[ulib][hypervisor] Support ARM mem traps.

We have a lot more information within an ARM mem trap packet, and we do
not need to decode any instructions, so we need to specialise the
handling of mem traps for ARM.

ZX-1347 #comment

Change-Id: I05847b2ef498d0f68fb49e95574439f54c376c68

[arm][hypervisor] Pass more data for mem traps.

When a mem trap occurs, we want to pass along more data in the packet so
that user-space can appropriately deal with it. Luckily, everything we
need is contained within the ISS of the exception syndrome, so we do not
have to do any instruction decoding.

ZX-1347 #comment

Change-Id: Ia550c63d2c22105cb36da2cd857cfd7742aa66eb

[arm][hypervisor] Prevent guest use of SMC.

Guests should be using HVC to make PSCI calls, so trap any attempt to
use SMC by a guest, and return an error.

ZX-1347 #comment

Change-Id: Idfde586de62efad5a62bd9dbbc3bac5df69f5dde

[arm][hypervisor] Fix handling of TLBI in EL2.

We should be using the proper value of VTTBR_EL2 when switching to the
guest in order to execute TLBI. Previously we were incorrectly using the
VMID, without combining it with the translation table address.

ZX-1347 #comment

Change-Id: I0fd3c4d291868ec0b4166e601d8be8f81cb75dfd

[arm64][hypervisor] Handle more VM sys registers

Add simple handling of MAIR_EL1, TCR_EL1, TTBR0_EL1, and TTBR1_EL1. We
simply apply the value to the system register, and re-enter the guest.
This is required as we are trapping all VM operations so we can properly
handle changes to SCTRL_EL1.

ZX-1347 #comment

Change-Id: Ibba9aafe68e368a133f7fffe065862966f2e8e2f

[arm64][hypervisor] Handle guest SCTRL_EL1.

When the guest enables or disables the MMU through SCTLR_EL1, we should
respond correctly by disabling or enabling default-cacheability. If both
HCR_EL2.DC and HCR_EL2.TGE are not 0, then setting SCTLR_EL1.M will have
no effect.

ZX-1347 #comment

Change-Id: Ic95be9988437ef3779fd3f997df864a25a5794a6

[arm64][hypervisor] Always handle page fault.

Even if ISS is not valid, we still have to handle the page fault when
there is a data abort.

ZX-1347 #comment

Change-Id: If4f8d60cad4d2951f0386b8489cdae9d3d05c0da

[arm64][hypervisor] Always map GPAS as RWX.

This matches what we do in x86, and avoids re-faulting for permission
changes.

Change-Id: I65eae864e5a25a98183c2cdab8643cd34632ee1f

[arm64][hypervisor] Handle page faults.

We still need to update the MMU code to invalidate the TLB when
unmapping pages for the balloon device, that will come in a separate CL.

ZX-1276 #done

Change-Id: I5ee00a84a39cb023fd566bfa92a7629ee2a0caed

[arm64][hypervisor] Add vcpu_{read,write}_state.

This allows us to read and write the general purpose registers of a
VCPU when execution is paused.

ZX-1271 #done

Change-Id: I057d951e35bb9134e69cd32914cab498853b0794

[arm64][hypervisor] Get VCPU running on hardware.

To get a VCPU running on the HiKey960, the following change were
required:
* Set default-cacheable, so that while the EL1 MMU is disabled, there is
not a memory type mismatch with the stage 2 translation table.
In the future, we'll to trap modification of SCTLR_EL1.M and turn off
default-cacheable is the MMU is enabled.
* Handle physical interrupts, and correctly return back to the EL1 host
so it can deal with the interrupt.
* Invalidate the VMID, so that subsequent uses of the VMID by other
guests do not use the previously cached TLB entries.

ZX-1241 #done

Change-Id: If30d6834ad05914bc66d1bb4dadc75e0abfd4c07

[arm64][hypervisor] Start basic VCPU execution.

While this works on QEMU, it fails with an EC of 0b000000 on the
HiKey960. It looks like the IPA isn't being calculated correctly, so
I've got to continue investigating that in a follow up CL.

ZX-1241 #comment In Progress

Change-Id: I3a97978b8fa3fc65166936ed2f02344edc30c427