[hypervisor][arm64] Trap guest debug reg access

To enable debugging on ARM64 we have to unlock
the OS Lock upon boot. We don't want the guest OS
to alter the state of the OS lock so we trap any
accesses to the debug registers and implement
them as WI/RAZ.

This should work as both Zircon and Linux unlock
the debug lock immediately once upon boot
regardless of whether or not the debug lock
is implemented.

An guest OS that attempts to query for the
presense of the debug lock will find that it is
unimplemented since the OSLSR is implemented as
RAZ.

Change-Id: Ifa42f4fa2a50fc3569eac9a194c19a33059d7151

[gicV3] GICv3 virtualization support, to allow zircon-guest to run on QEMU

PD-50 #Done

Change-Id: I60a6937ad9a0515d1d737fbe562be517eb9a66ef

[arm64][hypervisor] Combine TLBI HVC indices

Use only one HVC index for all TLBI operations in EL2, so that we
increase the space available in the exception table.

Change-Id: Idabcefd36986760198b1fff72031ca574e6793d8

[arm64][hypervisor] Simplify EL2 guest exit

If we're exiting due to an IRQ, we can skip some of the regular guest
exit logic.

Change-Id: I5e45ba3f96504b32013b2422b2b9db304f3ae290

[arm64][hypervisor] Fix FP/SIMD handling

With this fixed, Debian guests work perfectly and there are no more
translation faults.

PD-75 #done

Change-Id: I0bfe6b09936455fa984d7d4e405e16fb02899601

[arm64][hypervisor] Allocate a page for El2State

When we pass El2State from the host to EL2, we go from the host virtual
address to the 1:1 physical address mapping in EL2. This means that
EL2State can cross a page boundary unless we're careful, which would
cause us to write into the wrong memory if we're not careful.

Change-Id: I2ed2c0b0fe9c3981ac920f87e5307508cf3331ff

[arm64][hypervisor] Some small cleanups

1. Broadcast of some instructions to the inner-shareable domain.
2. Upgrade barriers to be at least inner-shareable.
3. Slight change to floating-point handling (still disabled).

Change-Id: I3c964ff1177459bc2d138dc3329652725e930773

[arm64][hypervisor] Disable EL2 alignment checking

Unaligned accesses in EL2 were causing exceptions. Disable alignment
checking.

Change-Id: Ic219239eefd28459bb3f44cf6dc26fda9fafd9c0

[arm64][hypervisor] Correctly align vector tables

The EL2 vector tables were misaligned, so when the address of the
secondary table was loaded into VBAR_EL2 (which has its first 10 bits
marked RES0), we were never able to load the secondary vector table.
This also meant that the primary vector table was prone to be misaligned
too.

This fixes both vector tables by aligning them correctly, and switching
to adr_global to load their addresses.

Change-Id: I8840ea1db6bd59b6fd88c46a2e317cb980711a98

[arm64][hypervisor] Disable EL2 FP/SIMD traps

Disable FP/SIMD traps until we discover why they're causing the
hypervisor to wedge on ARM64.

ZX-1685 #comment

Change-Id: I0a084fc08a2ae7239573c9d711674e966bf0bcc8

[arm64][hypervisor] Reset on invalid exception

Don't try to return to the host, just reset. Also add a few more static
assertions to el2_state.h to sanity check offsets.

Change-Id: Id8c4a727b655d66634a91cc3cd3b1cb02e6bf51c

[arm64][hypervisor] Set VMPIDR_EL2 based on VPID

This allows a VCPU to be assigned to any CPU core, without needing to be
assigned directly to the core with the same number as the VPID.

Change-Id: I659862bdd5479987c06667b64633d344b9b568fc

[arm64][hypervisor] Save/restore SPSR_EL1

Save and restore SPSR_EL1 when we do a world switch.

Change-Id: I631a54dd984aed39d445f0204d3020ea89ff11df

[arm64][hypervisor] Reset on invalid exception

When there's an exception we don't understand, reset the system.

Change-Id: Iccae2c49e252fec9c3a2cfe2e9de68fef2620061

[arm64][hypervisor] Handle unexpected EL2 exceptions

When we receive an unexpected EL2 exception, we should return back to
host with an internal error, rather than halt.

Change-Id: I31d549c0c2a61a10f931c5bd99cce5af4a2f288c

[arm64][hypervisor] Restore timer registers

Restore timer registers when we re-enter the guest. We still need to do
more to correctl handle timer interrupts, but that will come in a follow
up change.

Change-Id: Id8051ab164ddc2e1f83e476cf292a18b4ea323e2

[arm64][hypervisor] Misspelt elr_el1 in el2.S

When copying from another branch, I wrote esr_el1 instead of elr_el1.

Change-Id: Ieac3712b16d1e8e85f4e12d8110e44e0373aff14

[arm64][hypervisor] Save/restore ELR_EL1

What looked like reads from stale PTEs, turned out to be the lack of
save/restore of ELR_EL1. This allows us to re-enable paging in guest
physical memory.

ZX-1626 #done

Change-Id: I254808762c073d70dfa1d2353a78f0b555fae6dd

[arm64][hypervisor] Simplify use of el2_tlbi_ipa

Move the shifting logic into arm64_el2_tlbi_ipa so that it can be easily
used elsewhere. Also invalidate all stage-1 translations when we
invalidate a stage-2.

Change-Id: If352efb897aadbd23d687c2ea1fa5c4a84c1936a

[arm64][hypervisor] Reset virtual timer on exit

We should reset the virtual timer when we exit the guest, otherwise it
will fire when we are in the host.

Change-Id: Ia00d66f318b67d1c079b90d765a7bb7c0ac6495f

[arm64][hypervisor] Minor cleanup of el2.S

Change a few things to make it slightly clearer:
* Rename el2_mexec to el2_hvc_mexec
* Move the test for an FPU trap into el2_guest_exit (reducing
instructions in the vector table)
* Reorder the test for FPU state restore to be easier to read

Change-Id: Id9d1861356f545a30e869ca1e4c67e0e80b5a179

[arm64][hypervisor] Remove superfluous ISB

We don't need the ISB before altering FPU state.

Change-Id: I678845953b470fc1b4c66cad850e4f3a12e00e88

[arm64][hypervisor] Save/restore FPU state

Save and restore FPU state if the guest attempted to use the FPU.
Additionally save PAR_EL1 when doing a world-switch.

Change-Id: I162dbab429a669008a43a6a7ac687615fb7d918f

[arm64][hypervisor] Save guest virtual timer

Make sure we save the guest the virtual timer on VM exit. We then rely
on these saved values for timer handling, rather than on the current
value of the timer registers which may have been modified by the host.

Furthermore, this CL restructures the El2State and modifies the handling
of HostState in el2.S, so that we can increase the size of El2State
without hitting the limits of immediates in instructions.

Change-Id: I110397094be4f40bdce1ced25390554415c5d1c3

[arm64][mmu] Support TLBI of terminal stage-2 PTE

This matches the recent changes to the arm64 MMU code.

Change-Id: Iaea5c311f121744aac1511e317d8ab7d59d36dae

[arm64][hypervisor] Set timer on WFI

If the timer is enabled, set a timer before sleeping on a WFI. Once the
timer expires, we exit from sleep and re-enter the guest.

ZX-1347 #done

Change-Id: Ia2ec38e772876e4e64c4865b7a1369d2444eba98

[mexec][arm] Mexec to same EL that the system originally booted in.

Previously, mexec would always exec the new image at EL1, this change
makes mexec exec to the same EL that the system originally booted to
which should enable hypervisor development using zedboot.

Change-Id: I59c2556b74d653a90c5ec0367900d6ad5b8e0074

[arm64][hypervisor] Save and restore EL0 state

Add TPIDR_EL0, TPIDRRO_EL0, and SP_EL0 to SystemState, so that we save
and restore the correct guest state and host state.

This fixes the bug where TPIDR_EL0 is not correctly restored for the
host, so when we return to user-space and continue execution, we crash
as soon as TLS is used.

ZX-1347 #comment

Change-Id: If0912931e7060ce328766c16894883ef18be279b

[arm][hypervisor] Use FAR_EL2 for mem trap.

Combine the lower bits of FAR_EL2 with the HPFAR_EL2 to get the exact
address that cause a mem trap.

Without this change, we only had page-level granularity for a mem trap.

ZX-1347 #comment

Change-Id: I19aef7bd84ba78f7b6d43be7374399b3cc1cbba2

[arm][hypervisor] Fix handling of TLBI in EL2.

We should be using the proper value of VTTBR_EL2 when switching to the
guest in order to execute TLBI. Previously we were incorrectly using the
VMID, without combining it with the translation table address.

ZX-1347 #comment

Change-Id: I0fd3c4d291868ec0b4166e601d8be8f81cb75dfd

[arm64][hypervisor] Handle guest SCTRL_EL1.

When the guest enables or disables the MMU through SCTLR_EL1, we should
respond correctly by disabling or enabling default-cacheability. If both
HCR_EL2.DC and HCR_EL2.TGE are not 0, then setting SCTLR_EL1.M will have
no effect.

ZX-1347 #comment

Change-Id: Ic95be9988437ef3779fd3f997df864a25a5794a6

[arm64][hypervisor] Implement zx_vcpu_interrupt.

This implements interrupt injection for arm64. There's still more work
to be done when we start running Zircon or Linux on arm64.

I'll also port the new interrupt test to x86 in a follow up change.

ZX-1289 #done

Change-Id: I1a3ddaa428e7b249335533a0c65a601c900ce77c

[arm64][hypervisor] Only invalidate on first run.

Only invalidate the VMID TLB entries on the first run of a guest, to
protect against the case that that VMID has been used before.

Change-Id: I39c2b67ea3e8141631592e720d71f55e8a589af7

[arm64][hypervisor] Add TLB invalidation of guest.

Invalidate the TLB when modifying the guest physical address space PTEs.

ZX-1277 #done

Change-Id: I05d72850d7b54be53e8b98c65abd6c316c03bbf1

[arm64][hypervisor] Add vcpu_{read,write}_state.

This allows us to read and write the general purpose registers of a
VCPU when execution is paused.

ZX-1271 #done

Change-Id: I057d951e35bb9134e69cd32914cab498853b0794

[arm64][hypervisor] Get VCPU running on hardware.

To get a VCPU running on the HiKey960, the following change were
required:
* Set default-cacheable, so that while the EL1 MMU is disabled, there is
not a memory type mismatch with the stage 2 translation table.
In the future, we'll to trap modification of SCTLR_EL1.M and turn off
default-cacheable is the MMU is enabled.
* Handle physical interrupts, and correctly return back to the EL1 host
so it can deal with the interrupt.
* Invalidate the VMID, so that subsequent uses of the VMID by other
guests do not use the previously cached TLB entries.

ZX-1241 #done

Change-Id: If30d6834ad05914bc66d1bb4dadc75e0abfd4c07

[arm64][hypervisor] Setup MMU for EL2.

This is required to share the EL2 state between EL2 and the EL1 host in
a consistent way, so that the hypervisor has the correct view of the EL1
guest.

ZX-1241 #comment

Change-Id: I9b1c7a643c62550f498642095d9b6b63d9d8a99f

[arm64][hypervisor] Start basic VCPU execution.

While this works on QEMU, it fails with an EC of 0b000000 on the
HiKey960. It looks like the IPA isn't being calculated correctly, so
I've got to continue investigating that in a follow up CL.

ZX-1241 #comment In Progress

Change-Id: I3a97978b8fa3fc65166936ed2f02344edc30c427

[arm64][mmu] Add support for guest aspace.

Due to the way VTCR_EL2 is dependent upon ID_AA64MMFR0_EL1.PARange, and
the fact that PARange is limited to 40-bits on Cortex-A53, we can't use
the same configuration as the user-space mappings for guest mappings.

This change adds the basic setup for handling guest mappings, but
doesn't contain the TLBI operations needed. Those will come in a follow
up.

Change-Id: I9538ccc81a2c3d4afd141d2e28d3196a967fc370

[arm64][hypervisor] Setup VTCR_EL2.

This configures VTCR_EL2 to accept the same page tables as user-space.

Change-Id: I3a78d3bdcc59744886051f93e8064420b711658e

[arm64][hypervisor] Support host PSCI calls with EL2.

We now forward host PSCI calls to EL3. This change also makes non-KVM
arm64 run with virtualisation on.

Change-Id: Ie708d4a520aa67fc3f6f0899d3546ade322e0c8f

[arm64][hypervisor] Inline functions in vector table.

This reduces the size of the EL2 section slightly.

Change-Id: I1e437e979da145f89ca20f3e51ddaabe81a66ce5

[arm64][hypervisor] Setup initial VCPU state.

Before we can begin execution, we still need to setup VPIDR, VTCR, and
VTTBR. That will come in a follow up CL.

Change-Id: I307bc6c2ae3ca8af747afcd36d2616ea8e7419fd

[arm64][hypervisor] Setup EL2 translation table.

We setup TTBR0_EL2 so that it has an identity-mapped view of memory,
using the tt_trampoline.

Change-Id: I3e6d9c0125cde265d55bd344c8449d15ecd062b5

[zx] Magenta -> Zircon

The Great Renaming is here!

Change-Id: I3229bdeb2a3d0e40fb4db6fec8ca7d971fbffb94

[arm64][hypervisor] Add El2CpuState.

Add code to manage EL2 CPU state. Currently this does the very minimum
and sets the EL2 stack for each CPU.

Next, I'll start setting up more of the EL2 state and also guest
physical address space.

Change-Id: I18b7f9d00b236e52cdc317dffe3b42fcffbcb8fe