trivial: clean up code for C parser

Remove unused cases and add break in switch statements.
Add conditions to sel4arch.xml.
Change guard in capdl printing to correct TK1_SMMU.
Set KernelArmSMMU default to off.
Add types to aarch32 syscall_stub_gen.py.

Signed-off-by: Oliver Scott <Oliver.Scott@data61.csiro.au>

trivial: style and comment

Signed-off-by: Oliver Scott <Oliver.Scott@data61.csiro.au>

SMMU: assigning vspace to context banks

Supporting user-level applications to assign vsapce root to context
banks through system calls. This commit also configures the context
bank according to stage 1 or stage 2 requirement.

Signed-off-by: Oliver Scott <Oliver.Scott@data61.csiro.au>

SMMU: providing master control caps to root task

Adding the master control caps that are used to create transaction
and context banks caps. This commit includes the internal kernel
structure that required to manage any created transaction and
context bank caps.

Signed-off-by: Oliver Scott <Oliver.Scott@data61.csiro.au>

SMMU: basic driver for init and probing

Introducing the driver in kernel for detecting SMMU features
and initialise the hardware.

Signed-off-by: Oliver Scott <Oliver.Scott@data61.csiro.au>

Remove references to kernelBase

kernelBase was used inconsistently between different architectures to
refer to the either of the first kernel address or the first address of
the mappings of the kernel ELF region specifically.

These have been replaced with more consistent use of constants
explicitly describing which region is being referenced.

Signed-off-by: Curtis Millar <curtis.millar@data61.csiro.au>

Convert license tags to SPDX identifiers

This commit also converts our own copyright headers to directly use
SPDX, but leaves all other copyright header intact, only adding the
SPDX ident. As far as possible this commit also merges multiple
Data61 copyright statements/headers into one for consistency.

arm: Reserve VPPI and VGIC maintenance IRQs on SMP

This commit updates the secondary core initialisation to reserve
the VGIC maintenance and VTimer Event IRQs. This change is
introduced in support of SMP configurations when arm hypervisor
support is enabled.

arm_hyp: Save and restore vtimer state on switches

Added support for reading and writing additional virtual timer
registers for vcpu hw read and write accesses. These include the
compare value register (CNTV_CVAL) and offset register (CNTV_OFF),
each represented as two 32 bit (high and low) registers on aarch32 and
as single 64 bit registers on aarch64.

Added support for explicitly saving and restoring the virtual
timer registers when the vcpu is enabled and disabled. This
ensures when the vcpu is switched in and out, the virtual timer
registers are restored to a state that is consistent to when
it was last run.

By default the CNTVOFF register will be updated by the kernel to
accumulate the time the VCPU is not running. From the guest this will
result in the VCNT register not increasing when the VCPU is suspended.
This behavior can be turned off by disabling the
KernelArmVtimerUpdateVOffset config option.

arm,SMP: Refactor irq_t structure for smp

Explicitly create a struct definition for irq_t on SMP Arm
configurations. This makes it a lot harder to mistakenly use the wrong
irq encoding when moving an irq between a cnode index and hardware irq
number / core. A couple areas where this was being handled incorrectly
was fixed as part of the refactor. When performing an ipi for masking
PPI interrupts, the idx encoding is used as it fits into a single word.

Revert "Cache FPU enable state on all platforms"

This reverts commit 186180cf309b365b45c162f7a1f5931f0fc05d7c.

Revert "trivial: use global machine/fpu.h (arm)"

This reverts commit 89e53be40b1f94ed835d409a31229791c7f2b608.

trivial: use global machine/fpu.h (arm)

Cache FPU enable state on all platforms

This change broadens the caching of the FPU enable/disable bit from arm
to all platforms to ensure that on x86 time is not wasted on the
fastpath disabling or enabling an already disabled/enabled FPU.

boot: Use extra_bi_size_bits for vreg size calc

The bootinfo extra size region is rounded up to a power of 2 size before
being mapped into the initial threads address space. This wasn't being
handled correctly which lead to a situation where if the bootinfo
finishes on the last address of a page table but the extra region size
is not a power of 2 size then there won't be a page directory allocated
where one needs to be and the kernel would crash.

Revert the recent changes to Faster FPU switch

These changes need to be reverted until we can properly test the changes
with all of our hardware online (currently we cannot test on x86
hardware).

This reverts commit a0aeec3b304d16909619bc847197df221c73a0c4.
This reverts commit af0fdc82c23f1e7e3f1e568f6987f5afd0d2150a.
This reverts commit a190a14ecaae688a6b40dd0a699b8f49c2c0631e.

trivial: use global machine/fpu.h (arm)

Cache FPU enable state on all platforms

This change broadens the caching of the FPU enable/disable bit from arm
to all platforms to ensure that on x86 time is not wasted on the
fastpath disabling or enabling an already disabled/enabled FPU.

Create device untypeds at boot for all arches

Currently on x86 device untypeds are generated by passing the entire
address space minus any parts that are reserved by the kernel or that
are "real" memory (e.g. kernel image, physical RAM).

On ARM and RISC-V, device untypeds were generated at compile-time from
a device tree. This patch moves ARM and RISC-V to use the same approach
as x86, and moves the code from x86 into a common location that's
shared between the three architectures.

Co-Authored-By: Anna Lyons <anna@gh.st>

aarch64: allow access to memory below physBase

On aarch64 physBase is the constant that points to the bottom of
physical memory (RAM).

Prior to this change the kernel window was mapped directly to physBase,
which is usually not a 0 paddr. As a consequence the kernel could not
access any memory below physBase.

This change fixes this issue by mapping the start of the kernel window
to 0 in the physical address space.

- add new constant PADDR_LOAD, the location of the kernel image in the
physical address space.
- add new constant PADDR_BASE, the start of the physical address space
(0).
- add new constant KERNEL_ELF_BASE, the location of the kernel image in
kernel virtual memory.

A consequence of this change is that on aarch64, the kernelBase constant
now points to the start of the kernel window in virtual memory, but
*not* to the start of the kernel image as these are now different.

mcs: add periodic scheduling

This commit adds periodic scheduling with sporadic servers.

mcs: provide tickless api for arm timers

This does not implement the timers for any platforms, but
provides the generic arm arch, and aarch32/aarch64 infrastructure for
tickless timer drivers.

mcs: Add a scheduling context object

This is the first part of the seL4 MCS. This commit:

* adds a scheduling context object. Threads without scheduling
context objects cannot be scheduled.
* replaces tcbTimeSlice with the scheduling context object
* adds seL4_SchedControl caps for each core
* adds seL4_SchedControl_Configure which allows users to configure
amount of ticks a scheduling context has, and set a core for the
scheduling context.
* adds seL4_SchedContext_Bind, Unbind and UnbindObject, which allows
a tcb to be bound to a scheduling context.

arm,smp: setIRQState for each core timer

Setting the IRQState to IRQTimer for each per core timer prevents the
interrupts from getting masked the first time that they are received.

[SMP] Abstracted IRQ indexing to handle PPIs

irq_t is now a "virtual" interrupt type that encapsulates the
information of the core in case of a private interrupt. There is a
couple of macros that need to be defined on the interrupt controller
level to translate between virtual and hardware IRQs.

ARM boot: avoid variable + constant array access

This is workaround for part of binary verification, which currently
cannot handle some modes of array access.

boot: handle arm user image > kernel window

If we place the user image at the end of memory, the user image can be
beyond the kernel window. Handle this in arch_init_freemem by comparing
to paddr's not pptrs.

boot: allocate rootserver objects last

This change allows us to know, from just the kernel and dtb, where user
level untyped objects start being allocated from.

- allocate rootserver objects from last available freemem region.
- move create_rootserver_objects call into init_freemem.

arm: handle user image being at end of memory

The user image could be on either side of mode_reserved_region. Fix this
by assuming there is 1 or 0 mode_reserved_regions, and checking if the
user image is either side.

boot: refactor allocation of rootserver objects

Prior to this change, the boot process would dynamically allocate
memory for root server objects based on the order of initialisation.
Allocation was a best-fit algorithm.

This change preallocates all memory for root server objects to an
aligned untyped just after the user image. By allocating the objects in
order of size, allocation is greatly simplified and the ability to
reproduce the allocation offline based on the kernel and user image
sizes is increased.

boot: fix dtb region size calculation in arm

Include padding to round up to a page boundary.

arm: remove plat/machine/timer.h

plat/machine/timer.h is unused on arm, so remove it and fix up the
includes.

arm: clean & consolidate pmu management code

- consolidate armv_init_ccnt into arm_init_ccnt, which is general
- move armv_handleOverflowIRQ definition to benchmark.h
(no need for extra header)
- clean up definition of armv_handleOverflowIRQ

arm: make mode reserved regions a constant

Replace an unneccessary function call with a constant. This simplifies
and increases the maintainability of arch_init_freemem.

boot: consolidate init_freemem

init_freemem sets up available regions of memory for the hardware that
seL4 is booting on. Previously this function was duplicated across
architectures, with minor duplication.

This change provides a top-level init_freemem suitable for all
architectures.

- arm/riscv: change get_avail_p_reg to return whole p_reg array
- update all architectures to use new init_freemem.
- wrap init_freemem calls with arch_init_freemem for each arch, where
arch_init_freemem sets up the available and reserved regions of memory
which are passed to init_freemem.

boot: allocate irq cnode statically

The memory used for the irq cnode is never available to the user. As a
result this memory can be allocated statically, simplifying the
bootcode.

- remove allocation of irq cnode
- add static init
- generate irq cnode size from cmake for arm
- add static constants for riscv, x86 as there is no variability at the
moment.

ARM: optionally pass bootloader DTB to userspace

We now support receiving a DTB from the ELF loader and passing it on
to userspace in extra bootinfo. We still support booting without a DTB,
though - the device tree address is set to zero in the boot code and no
extra bootinfo region is provided.

aarch64: adjustments to pass c-parser

This commit updates the aarch64 build to pass the c-parser.

- replace 0b constants with decimal
- remove empty array definition
- replace __uint128_t with uint64_t, and double the array size
- remove variable shadowing

No further verification guarantees are provided for aarch64 code.

style: set min-conditional-indent to 0

Given we use braces all the time conditional indents do not make code
cleaner.

style: consistently attach return type

Add attach-return-type to astyle

style: consistently align pointer with name

Run astyle with align-pointer=name

style: correct parenthesis padding

Use astyle's unpad-paren to unpad all parentheses that are not included
by pad-header, pad-oper, and pad-comma.

arm: Rename plat_cleanInvalidateCache

Rename plat_cleanInvalidateCache to plat_cleanInvalidateL2Cache to
indicate that it is a L2 cache maintenance operation only.

arm: Assert that device regions don't overlap

The assert has an assumption that regions are also ordered in ascending
order.

TX1: error on SMP + cpu utilisation tracking

TX1 has multiple interrupts for the PMU, one per core. This is
not currently implemented, and is tracked in SELFOUR-1252. It will
work on 1 core.

armv8: Check TCR_EL2 configuration in init_cpu.

armv8: Add SMP support for aarch64

(1) Use NODE_STATE() to access per-core kernel data.
(2) Allocate kernel stack in src/arch/arm/64/head.S.
(3) Use the TPIDR_EL1 to contain the kernel stack pointer
as well as the logical core ID. The kernel stack must
be 4-KiB aligned, and the lowest 12 bits of TPIDR_EL1
are for the logical core ID.
(4) Define the LD_EX, ST_EX, and OP_WIDTH as "ldxr", "stxr",
and "w".
(5) Add irq_remote_call_ipi and irq_reschedule_ipi

Make TIMER_CLOCK_HZ available in timer.h for ARM

The RT kernel needs this value accessible to common functions.

Move definition of initTimer to be arm specific

initTimer does not exist on x86

arm: Add missing memory fence

Without a release+acquire fence here the compiler may assume that the value of ksNumCPUs
does not change, and essentially elide the loop.

Share linker.h between architectures

SMP: Introduce ENABLE_SMP_SUPPORT

- Make it more readable and less confusing compared to the 'CONFIG_MAX_NUM_NODES > 1' check

SELFOUR-806: aarch32 - Implement lazy FP save/restore

This commit implements lazy FP save/restore mechanism, required for user threads
when they use the FPU. The following caveats take place:

* Only support synchronous exceptions. No support for asynchronous ones.
* VFP opcodes are used instead of normal instructions to discard compiler warnings/errors.
* Support is limited to specific ARM subarchitectures we support and tested this commit on.
* Disable the FPU by default if users are not using it to avoid channels.
* FPU support is not verified yet.
* Will not work properly if using a VM that's running besides other VMs/threads that are using the FPU.

SELFOUR-746: IPI Handler for ARM

ARM/SMP: rename/add IRQ macros to match with x86

SELFOUR-748: ARM - Support local/remote TLB invalidation operations

smp: fix bugs when stalling remote core

- Restart TCB from inside the lock if it is waiting for anything other than IRQ
- Only replace the TCB with idle thread if it is in ThreadState_RunningVM state

Also, this makes the design generic to be shared with arm.

trivial: typos in previous commit

refactor: initialise common core state in one place

[STYLE_FIX]

SELFOUR-744: ARM - preliminary booting of other cores

ARM/IRQ: add a function to initialise local IRQ controller for CPUs

Similar to initIRQController(), cpu_initLocalIRQController() has to
be implemented for each platform. For those that are gonna support
SMP, it should be implemented proprely (if required), otherwise
it can be stubbed for single core implementations.

Remove assumption on 32-bit memory offsets

On 64-bit platforms physical address could be >2^32 offset from a virtual address. This
changes offsets to match the word size of the target architecture

ARM/Boot: initialise private timer, user access and benchmark in CPU context

In SMP context, init_cpu is meant to be called for each CPU, where
initialising private timer, user access and benchmarks (basically CCNT)
are per-CPU operations and involve per-CPU hardware
resources/interrupts.

aarch64: FPU lazy context switch

Verification: Guard getCurrentCPUIndex() with SMP #define

[SELFOUR-744]: Modify head.S to load stacks for each core and implement getCurrentCPUIndex

[STYLE_FIX]

Update init_freemem to check for mode dependant reserve regions

aarch64: Initial implementation

ARM: Refactor how we get/set seL4 kernel stack

This commit makes seL4 hold the stack address on TPIDRPRW register (PL1 only
Thread ID Register, VMSA, see B4.1.150 section of ARMv7-A and ARMv7-R
edition reference manual).

Originally, the kernel stack was (re-)mapped to a fixed kernel virtual address and
on traps sp is loaded with this virtual address. Reason is to shave up some
cycles/instructions on (fast) traps path instead of having two instructions
(loading global variable pointer and then memory-load stack address from
it).

With ARM pipelining getting smart/complex (from ARMv7 onwards), there is
no noticeable difference in performance between the two implementations.
Furthermore, this change makes it easy for SMP to load different stacks
for each core and do CPU ID arithmetic efficiently. It also avoids the
issues involving (re-)mapping stacks for different cores if we
followed the original design.

Note: the main side-effect to this change is that the kernel stack for
ARM is no longer mapped with execute-never attribute.

SELFOUR-499: X86, ARM: Add userspace invocations for hardware debugging

This commit implements the body of SELFOUR-499. The API exposes the x86 DR0-7
and ARM coprocessor 14 features to userspace by virtualizing them as context-
switched registers in the TCB. Implemented as TCB invocations. This feature is
only built when CONFIG_HARDWARE_DEBUG_API is selected.

* Add low-level support routines for setting, unsetting, getting, enabling
and disabling breakpoints.
* Add support for single-stepping as well.
^ Single-stepping is not supported on ARMv6 since the hardware
doesn't have support.
^ ARM implements single-stepping as instruction breakpoints
configured to fault on every instruction -- this is achieved through
the "mismatch" mode, which is only supported from ARMv7 onwards.
* Also support explicit software break requests, a la "BKPT" and "INT $3".

* New invocations:
* seL4_TCB_SetBreakpoint().
* seL4_TCB_GetBreakpoint().
* seL4_TCB_UnsetBreakpoint().
* seL4_TCB_ConfigureSingleStepping().
* New constants:
^ Event types:
^ seL4_InstructionBreakpoint.
^ seL4_DataBreakpoint.
^ seL4_SoftwareBreakRequest.
^ Access types:
^ seL4_BreakOnRead.
^ seL4_BreakOnWrite.
^ seL4_BreakOnReadWrite.
^ Exports:
^ seL4_NumHWBreakpoints.
^ seL4_NumExclusiveBreakpoints.
^ seL4_NumExclusiveWatchpoints.
^ seL4_NumDualFunctionMonitors.
^ seL4_FirstBreakpoint.
^ seL4_FirstWatchpoint.
^ seL4_FirstDualFunctionMonitor.

See documentation in the seL4 API manual.

SELFOUR-421 Introduce explicit device frames and untypeds

Kernel objects cannot be created from device untypeds, with the
exception of frames, which do not get zeroed and cannot be used
as an IPC buffer. Device untypeds additionally cannot be used
in the construction of ASID pools.

This then changes the API to the rootserver (i.e. bootinfo) to
send device untypeds instead of device frames. On ARM these
device untypeds are the same as the previously exported device
frame regions. On x86 PCI scanning is removed and all physical
memory addresses (that are not important for kernel integrity)
are released to the user.

In order to have bits in the frame and untyped caps on ARM the
number of software ASIDs had to be reduced from 2^18 to 2^17,
and the maximum untyped size reduced from 2^31 to 2^30

Provide generic C entry/exit hook routines

It can be deseriable to run code before/after user mode, but
not have to write it in assembly. This commit adds such stubs
that get called as the first/last C code when coming in
and out of the kernel

SELFOUR-551: Remove direct accesses to machine data structures

This commit wraps accesses to the vgic in functions that are marked as
DONT_TRANSLATE and uses a global constant (that is determined at boot time)
for the number of LR registers to prevent loops that have undefined bounds

SELFOUR-448 Benchmark: Track thread's CPU utilisation time

Refactor duplicated code in hardware.h

One copy to rule them all.

arm-hyp: Minor cleanups

Cleanup some small licensing, whitespace and configuration details

SELFOUR-229: add error condition for root task overlapping kernel window on ARM

arm-hyp/tk1: add guards so other plats can compile.

arm-hyp/tk1: make style

arm-hyp/tk1: address review comments.

SELFOUR-446 Benchmark: Track syscall feature

Benchmark feature that currently:
- Keeps track of system calls info
- Start time
- Duration
- Capability type
- Invocation tag
- Log the number of invocations of each system call*
- Log the number of invocations for each capability type per syscall.
- Has 3 new syscalls (dump, reset, get size).
- This new feature uses the existing log buffer (which is 1MiB for x86
and ARM). Since the number of syscall invocations is not
deterministic, the logged number of invocations is limited by the size
of the buffer. I suggested to enable the users to pass their own
user-level buffer, to enable more flexibility, later.
- ENABLE_BENCHMARKS is now a parent config option of trace points and
system call track features, they can't be used at the same time.

arm-hyp/tk1: reserve the SMMU interrupt

arm-hyp/tk1: put iospace caps to bootinfo

arm/tk1: a checkpoint for SMMU implementation

arm_hyp: add file to contain hyp mode inline funcs

machine_pl2.h:
new place for hyp mode inline functions.

boot.c:
replace #ifdef ARM_HYP with config_set(ARM_HYP).

vcpu.h, machine_pl2.h:
add empty functions when ARM_HYP is not defined to
pass compilation.

arm_hyp: add file to contain hyp mode inline funcs

machine_pl2.h:
new place for hyp mode inline functions.

boot.c:
replace #ifdef ARM_HYP with config_set(ARM_HYP).

vcpu.h, machine_pl2.h:
add empty functions when ARM_HYP is not defined to
pass compilation.

SELFOUR-114: remove bootinfo.h duplication

Remove mode dependant codes for initial thread from architecture dependant boot codes

Do not switch to the the root thread until after cpu initialization is done

Change additional int->word_t due to interraction with the C parser to ease verification

Rename uint32_t -> word_t in any relevant places

Removed CONFIG_BENCHMARK config option. All checks for '#ifdef CONFIG_BENCHMARK' replaced with '#if CONFIG_MAX_NUM_TRACE_POINTS > 0'.

ARM: Map the initial thread's IPC buffer and boot info as non-executable.

Note that this also required some irrelevant x86 changes to match function
arguments.

ARM: Add config options for exporting PMU and ARM generic timer physical count registers to PL0

arm: Rename BI_CAP_IT_PD to BI_CAP_IT_VSPACE to match x86 and common boot code

arm: Perform additional cache clean to prevent TLB walker loading garbage entries for first user process

Recommit of arm_hyp branch on release snapshot

Release snapshot