MFC r305094, r305096-r305097

r305094 by cognet:
Garbage collect bits forgotten in r295267.

r305096 by cognet:
Some old arm ports don't load the kernel at the beginning of the memory,
because the bootloader, ie redboot, won't let them do so, and so used the
memory before the kernel for early memory allocation, such as pagetables,
stacks, etc...
Make a bit of an effort to try to get that memory mapped.

r305097 by cognet:
Nuke obio_bs_tag, it was used before it was initialized, and
arm_base_bs_tag is the same, anyway.

MFC r320056: arm: set appropriate section flags for .init_pagetable

The arm kernel linker scripts place the .init_pagetable section in .bss,
but .init_pagetable had no section flags set, and so did not match the
expected flags for .bss.

GNU ld silently ignores this case, but lld reports an error:

ld: error: incompatible section flags for .bss
>>> locore.o:(.init_pagetable): 0x0
>>> output section .bss: 0x3

PR: 220055
Sponsored by: The FreeBSD Foundation

MFC r312292, r313573:

Stop including sys/types.h from arm's machine/atomic.h, fix the places
where atomic.h was being included without ensuring that types.h (via
param.h) was included first, as required by atomic(9).

Remove arm's cpuconf.h, and references to it, after moving a few lines from
it into pmap-v4.h where they are used. Other than those few lines of
support for different MMU types, nothing in cpuconf.h has been used in our
code for quite a while.
The file existed to set up a variety of symbols to describe the
architecture. Over the past few years we have converted all of our source
to use the new architecture symbols standardized by ARM Inc, and predefined
by both clang and gcc.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Use the new(-ish) CP15_SCTLR macro to generate system control reg accesses
where possible. In the places that doesn't work (multi-line inline asm,
and places where the old armv4 cpufuncs mechanism is used), annotate the
accesses with a comment that includes SCTLR. Now a grep -i sctlr can find
all the system control register manipulations.

No functional changes.

Rename pte.h to pte-v4.h and start including directly either pte-v4.h
or pte-v6.h in files which needs it.

There are quite internal definitions in pte-v4.h and pte-v6.h headers
specific for corresponding pmap implementation. These headers should be
included only in very few files and an intention is to not hide for
which implementation such files are.

Further, sys/arm/arm/elf_trampoline.c is an example of file which
uses armv4 like pmap implementation for both armv4 and armv6 platforms.
This is another reason why pte.h which includes specific header
according to __ARM_ARCH is not created.

Allow armv4/5 kernels to be loaded on any 2MB boundary, like armv6/7.

This eliminates the reliance on PHYSADDR and KERNPHYSADDR compile-time
symbols (except when the rom-copy code is enabled) by using the current
PC and the assumption that the entry-point routine is in the first 1MB
section of the text segment.

Other cleanups done:

- Reduce the initarm() stack size back to 2K. It got increased to
4 * 2K when this file was supporting multicore armv6, but that
support is now in locore-v6.S.

- When building the temporary startup page tables, map the entire
4GB address space as VA=PA before mapping the kernel at its loaded
location. This allows access to boot parameters stored somewhere
in ram by the bootloader, regardless of where that may be.

- When building the page table entry for supporting EARLY_PRINTF, map
the section as uncached unbuffered, since it is presumably device
registers.

Note that this restores the ability to use loader(8)/ubldr on armv4/5
kernels. That was broken in r283035, the point at which ubldr started
loading an arm kernel at any 2MB boundary.

Also note that after this, there is no reason to set KERNVIRTADDR to
anything other than 0xc0000000, and no need for PHYSADDR or KERNPHYSADDR
symbols at all.

Fix the spelling of __ARM_ARCH >= 6 in sys/arm/arm.

Cleanup a little more:
- Remove whitespace at the end of lines
- Use a tab after instructions, not spaces

Fix the style of locore-v4.S and locore-v6.S to help find any common code.

Remove the armv6 code from locore-v4.S, it's not needed there.

Remove the SMP code from locore-v4. These will never use the SMP code as
there is no multi-core hardware prior to ARMv6.

Sponsored by: The FreeBSD Foundation

Fix alignment directives in arm asm code after clang 3.5 import.

The ancient gas we've been using interprets .align 0 as align to the
minimum required alignment for the current section. Clang's integrated
assembler interprets it as align to a byte boundary. Fortunately both
assemblers interpret a non-zero value as align to 2^N so just make sure
we have appropriate non-zero values everywhere.

Rename locore.S to locore-v4.S and add a new locore-v6.S for starting up
armv6/7 systems. We need to use some new armv6/7 features at startup
and splitting the implemenations to separate files will be more maintainable
than adding even more #ifdef sections to locore.S.

Because of the standardized interfaces to cache and MMU operations in armv7,
we can tolerate the kernel being entered with caches enabled. This allows
running u-boot and loader(8) with caches enabled, and the performance
improvement can be dramatic (boot times can be cut from over a minute
to under 30 seconds). The new implementation also has more robust cache
and mmu sequences for launching AP cores, and it paves the way for
upcoming changes to the pmap code which will use the TEX remap feature.

Changes in mp_machdep.c work with the new behavior in locore-v6 mp_entry,
and also reuse the original boot-time page tables to get transitioned
from physical to virtual addressing before installing the normal tables.

Submitted by Svatopluk Kraus and Michal Meloun with some changes by me.

Fix the name of the coprocessor to include the "p" prefix, the clang
integrated assembler expects this.

MFC after: 1 Week
Sponsored by: ABT Systems Ltd

Allow the armv6 kernel to be build with PHYSADDR undefined. The kernel
will now find the virtual to physical mapping for libkvm to use at
runtime. This makes PHYSADDR redundant, however keep it around to give
everyone a chance to update their libkvm.

MFC after: 1 week

Unify interrupts bit definition and usage. While here remove PSR_C_bit.

Submitted by: Svatopluk Kraus <onwahe at gmail.com>,
Michal Meloun <meloun at miracle.cz>
Differential Revision: https://reviews.freebsd.org/D754

When entering the kernel with the MMU off assume we are running from a
va == pa map.

I'm not sure the code would work if we are not running from the identity
map as the ARM core may attempt to read the next instruction from an
invalid memory location.

Remove Lvirtaddr and Lphysaddr, these don't appear to be used.

Generalise the va to pa code and use it when starting secondary cores

Reviewed by: ian@, rpaulo@
Differential Revision: https://reviews.freebsd.org/D736

Fix the handling of MMU type in the AP entry code. The ARM_MMU_V6/V7
symbols are always #defined to 0 or 1, so use #if SYM not #if defined(SYM).
Also, it helps if you include the header file that defines the symbols.

Fix unwind-info errors in our hand-written arm assembler code.

We have functions nested within functions, and places where we start a
function then never end it, we just jump to the middle of something else.
We tried to express this with nested ENTRY()/END() macros (which result
in .fnstart and .fnend directives), but it turns out there's no way to
express that nesting in ARM EHABI unwind info, and newer tools treat
multiple .fnstart directives without an intervening .fnend as an error.

These changes introduce two new macros, EENTRY() and EEND(). EENTRY()
creates a global label you can call/jump to just like ENTRY(), but it
doesn't emit a .fnstart. EEND() is a no-op that just documents the
conceptual endpoint that matches up with the same-named EENTRY().

This is based on patches submitted by Stepan Dyatkovskiy, but I made some
changes and added the EEND() stuff, so blame any problems on me.

Submitted by: Stepan Dyatkovskiy <stpworld@narod.ru>

For old CPUs, map the 64 first MB of RAM as it used to be. Some ports
(XScale mainly) expects the memory located before the kernel to be mapped,
and use it to allocate the page tables, the various stacks, etc.
A better fix would probably be to rewrite the various bla_machdep.c to stop
using that RAM, but I'm not so inclined to do it, especially since I don't
have hardware for all of them.

Fix spelling mistake in comment.

Spotted during: http://www.bsdcan.org/2014/schedule/events/484.en.html

Call idcache_inv_all from the AP core entry code before turning on the MMU.
Also, enable instruction and branch caches, which should be safe now that
they're properly initialized/invalidated first.

Consolitate all the AP core startup stuff under a single #ifdef SMP block.
Remove some other ifdefs that came in with a copy/paste that mean basically
"if this processor supports multicore stuff", because if you're starting up
an AP core... it does.

Move the mptramp code which is specific to the Marvell ArmadaXP SoC out of
the common locore.S file and into the mv/armadaxp directory.

Fix the arm sys_sigreturn(): its argument is a struct ucontext, not a
struct sigframe containing the struct ucontext.

The signal trampoline return code on the other hand DOES have just a
struct sigframe on the stack to work with, so have it get a pointer to
the ucontext out of there to pass along to sys_sigreturn.

In other words, make everything work right whether sys_sigreturn is
invoked from the trampoline or from userland code calling sigreturn(2).

Submitted by: Takashi Komatsu <komatsu.taka@jp.panasonic.com>
Reviewed by: cognet

Actually set the proper bit to indicate TTB shared memory.

Submitted by: Juergan Weiss

Allow the kernel to be loaded at any 1MiB address. This requirement is
because we use the 1MiB section maps as they only need a single pagetable.

To allow this we only use pc relative loads to ensure we only load from
physical addresses until we are running from a known virtual address.

As a side effect any data from before or 64MiB after the kernel needs to
be mapped in to be used. This should not be an issue for kernels loaded
with ubldr as it places this data just after the kernel. It will be a
problem when loading directly from anything using the Linux ABI that
places the ATAG data outside this range, for example U-Boot.

Swap PA and VA so they are in the right registers...

Pass the pagetable used from locore.S to initarm to allow it to map data
in as required.

Remove the now unused MMU_INIT macro.

Dynamically generate the page table. This will allow us to detect the
physical address we are loaded at to change the mapping.

Pass the kernel physical address to initarm through the boot param struct.

Update all arm code that manipulates the PSR registers to use modern syntax.

It turns out the version of gas we're using interprets the old '_all' mask
as 'fc' instead of 'fsxc'. That is, "all" doesn't really mean "all".

This was the cause of the "wrong-endian register restore" bug that's
been causing problems with some cortex-a9 chips. The 'endian' bit in the
spsr register would never get changed (it falls into the 'x' mask group)
and the first return-from-exception would fail if the chip had powered on
with garbage in the spsr register that included the big-endian bit. It's
unknown why this affected only certain cortex-a9 chips.

Fix silly typo...

Remove STARTUP_PAGETABLE_ADDR from the ARM configs and replace it with
memory at the end of the kernel.

This helps reduce the SoC and board specific configuration required.

Reviewed by: bsdimp
Tested by: jmg (armeb), br

Add support for mapping a small range of the SoC devices for debugging
purposes early in boot.

Add identification and necessary type checks for Krait CPU cores. Krait CPU is used in
Qualcomm Snapdragon S4 and Snapdragon 400/600/800 SoCs and has architectural
similarities to ARM Cortex-A15. As for development boards IFC6400 series embedded
boards from Inforce Computing uses Snapdragon S4 Pro/APQ8064.

Approved by: stas (mentor)

Enable missing Access Flag for secondary cores on ARMv6/v7

Spotted by: Wojciech Macek <wma@semihalf.com>
Obtained from: Semihalf

> Description of fields to fill in above: 76 columns --|
> PR: If a GNATS PR is affected by the change.
> Submitted by: If someone else sent in the change.
> Reviewed by: If someone else reviewed your modification.
> Approved by: If you needed approval for this commit.
> Obtained from: If the change is from a third party.
> MFC after: N [day[s]|week[s]|month[s]]. Request a reminder email.
> Security: Vulnerability reference (one per line) or description.
> Empty fields above will be automatically removed.

M sys/arm/arm/locore.S

Remove hard-coded mappings related to Armada XP support

Armada XP initialization flow requires SoC registers to be
mapped very early in order to configure Snoop Filter for SMP.
Additional mapping in locore.S is redundant as proper mapping is
made in pmap_devmap_bootstrap() prior to calling cpu_setup() which
configures the Snoop Filter.
For secondaru CPUs it is better to pass VA of the SoC
registers defined in MV_BASE and PA consistent with the value
in the Device Tree.

Tested by: kevlo

Fix a register name typo. The effect was that CPU_CONTROL_AFLT_ENABLE
wasn't being set, but it was almost assuredly already turned on anyway
by the bootloader.

Switch to AP[2:1] access permissions model. Store "referenced"
bit in PTE.

Enable Access Flag in CPU control. With AF enabled each valid mapping
needs to have referenced bit in PTE set in order to be able to cache
it in the TLB.

AP[0] bit is to be used as reference flag.
All access permissions are encoded by AP[2:1] wherein AP[1] is in fact
"user enable" and AP[2](APX) is "write disable".

All mappings are always set to be valid. Reference emulation is performed
by setting/clearing reference flag in PTE.

md.pvh_attrs are no longer necessary however pv_flags are still being used
for now.

Marking vm_page as "dirty" or "referenced" is being performed on:
- page or flag fault servicing in pmap_fault_fixup(), basing on the fault
type
- vm_fault servicing in pmap_enter() according to the desired protections
and faulty access type
Redundant page marking has been removed as on ARM we know exactly when the
particular page is referenced or is going to be written.

Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Sponsored by: The FreeBSD Foundation, Semihalf

Correct comment about initial VA=>PA mapping

Properly initialize Armada XP MP subsystem.

- correct setting of Auxiliary Control Register for MP mode
- correct setting of Auxiliarty Debug registers
- cleanup management of memory contains bootup code
- early initialization of Coherency Fabric (MP and not-MP mode)
- enable Snoop Filtering

Obtained from: Semihalf

Insert STOP_UNWINDING directives in the _start (kernel entry point) and
fork_trampoline (thread entry point) assembler routines, because it's
not possible to unwind beyond those points.

Also insert STOP_UNWINDING in the exception_exit routine, to prevent an
unwind-loop at that point. This is just a stopgap until we get around
to instrumenting all assembler functions with proper unwind metadata.

When running on armv6, set alignment checking to modulo-4 mode rather
than modulo-8, because clang emits ldrd and strd instructions for
addresses that are only 4-byte aligned.

Add an END macro to ARM. This is mostly used to tell gas where the bounds
of the functions are when creating the EABI unwind tables.

Ensure the stack is correctly aligned before calling the first C function.

Update sigcode to use both the current ABI and FreeBSD's version of the
ARM EABI syscall calling convention.

The current ABI encodes the syscall number in the instruction. This causes
issues with the thumb mode as it only has 8 bits to encode this value and
we have too many system calls and by using a register will simplify the
code to get the syscall number in the kernel.

With the ARM EABI we reuse the Linux calling convention by storing the
value in r7. Because of this we use both methods to encode the syscall
number in this function.

Replace generic ARM11 option with more specific
support for ARM1136 and ARM1176

Submitted by: Daisuke Aoyama <aoyama at peach.ne.jp>
Obtained from: NetBSD

Do not enable data cache until later in kernel init. Stale bits in
cache might cause erroneus behavior on early stage.

Submitted by: Ian Lepore
Tested on: Atmel, Marvell, and Eyxnos

Merging projects/armv6, part 1

Cummulative patch of changes that are not vendor-specific:
- ARMv6 and ARMv7 architecture support
- ARM SMP support
- VFP/Neon support
- ARM Generic Interrupt Controller driver
- Simplification of startup code for all platforms

Minor rearrangement of the locore <-> initarm interface. Pass in a
structure with the first 4 registers to allow a wider range of boot
loaders to work. Future commits will make use of this to centralize
support for the different loaders.

Remove unused cruft. We call through memcpy more directly when we
need to move the kernel, so we no longer need this.

This comment has become unmoored from the code to which it applies.
Move it back.

Accept r1 as having the metadata pointer argument if r0 is 0.
This provides backward compatibility with Juniper loaders.

Sponsored by: Juniper Networks

Disable interrupts to allow booting on firmware (e.g. U-Boot) that
has interrupts enabled and active.

Obtained from: Juniper Networks, Inc.

Use Oxf0000000 instead of 0xff000000 to guess the physical address, relative
to the virtual one. I may had a reason at some point to use the later, but
can't remember which, and it can leads to issues.

Reported by: Guillaume Ballet <gballet gmail com>

Initial support of loader(8) for ARM machines running U-Boot.

This uses the common U-Boot support lib (sys/boot/uboot, already used on
FreeBSD/powerpc), and assumes the underlying firmware has the modern API for
stand-alone apps enabled in the config (CONFIG_API).

Only netbooting is supported at the moment.

Obtained from: Marvell, Semihalf

One more L2 cache synchronization call that didn't make the previous commit.

Remove some long-dead code

Reviewed by: cognet

ARM locore cosmetics.

Approved by: cognet (mentor)

Fix a comment to reflect the truth.

Spotted out by: Marius Nuennerich <marius.nuennerich AT gmx D0T de>
Approved by: re (blanket)

Erm we can't change the value of arm_memcpy if we're running from flash.
Instead, make memcpy() check if we're running from flash, and avoid
using arm_memcpy if we're doing so.

Teach the kernel and the ELF trampoline how to boot from onboard flash.

Nitsville: the routine is called initarm, not init_arm, correct it in
a comment.

Ooops, I removed the wrong bits.
This unbreak boot from a VA which is different from the PA.

- Disable the instruction cache very early, until it's time to enable it again.
- Revamp the code that jumps from physical to virtual address.

Export the virtual and physical address in which the kernel was loaded,
needed for userland when reading kernel dumps.

fix up white space, I had a simple comment fix, but I might as well do the
rest while I'm here...

MFp4: get the code that set the pc correctly to work, remove a few IQ31244
specific mappings from locore.S, re-organize iq31244_machdep.c to work with
the new locore.S

Spotted out by: jmg

Make sure we can boot both with and without MMU enabled.

Start all license statements with /*-

Use the RET macro.

If we're still running at the physical address, jump to the virtual address
instead before calling initarm().
This removes the need to map virtual == physical in initarm().

Remove dead code.

Map the kernel very early if needed.
Implement sigcode.

Remove filename+line number from panic messages.

Import FreeBSD/arm kernel bits.
It only supports sa1110 (on simics) right now, but xscale support should come
soon.
Some of the initial work has been provided by :
Stephane Potvin <sepotvin at videotron.ca>
Most of this comes from NetBSD.