MFC r325438:

All CP15 registers are bit fields or counters, don't use signed type when
accessing them.

MFC r306756:

ARM: SEV/WFE instructions are implemented starting from ARMv6K, use it
directly.

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

Include machine/acle-compat.h in cdefs.h on arm if the compiler doesn't
have ACLE support built in. The ACLE (ARM C Language Extensions) defines
a set of standardized symbols which indicate the architecture version and
features available. ACLE support is built in to modern compilers (both
clang and gcc), but absent from gcc prior to 4.4.

ARM (the company) provides the acle-compat.h header file to define the
right symbols for older versions of gcc. Basically, acle-compat.h does
for arm about the same thing cdefs.h does for freebsd: defines
standardized macros that work no matter which compiler you use. If ARM
hadn't provided this file we would have ended up with a big #ifdef __arm__
section in cdefs.h with our own compatibility shims.

Remove #include <machine/acle-compat.h> from the zillion other places (an
ever-growing list) that it appears. Since style(9) requires sys/types.h
or sys/param.h early in the include list, and both of those lead to
including cdefs.h, only a couple special cases still need to include
acle-compat.h directly.

Loves it: imp

Add four functions which check a virtual address for stage 1 privileged
(PL1) and unprivileged (PL0) read/write access. As cp15 virtual to
physical address translation operations are used, interrupts must be
disabled to get consistent result when they are called.

These functions should be used only in very specific occasions like
during abort handling or kernel debugging. One of them is going to be
used in pmap_fault(). However, complete function set is added. It cost
nothing, as they are inlined.

While here, fix comment of #endif.

Reviewed by: kib

Remove redundant __ARM_ARCH >= 6 check in armv6 specific files.

ARM: Use new ARMv6 naming conventions for cache and TLB functions
in all but ARMv4 specific files.
Expand ARMv6 compatibility stubs in cpu-v4.h. Use physical address
in L2 cache functions if ARM_L2_PIPT is defined.

ARM: Introduce new cpu-v4.h header and move all ARMv4 specific code
from cpu-v6.h to it.
Remove unneeded cpu-v6.h includes.

Introduce support for HW watchpoints and single stepping for ARMv6/v7

Allows for using hardware watchpoints for 1, 2, 4, 8 byte long addresses.
The default configuration of watchpoint is RW but code allows to select
RO or WO and X.
Since debugging registers are per-CPU (CP14) the watchpoint is set on
the CPU that was lucky (or not) to enter DDB.

HW breakpoints are used to perform single step in KDB.
When HW breakpoint is enabled all watchpoints are temporary disabled
to avoid recursive abort on both watchpoint and breakpoint.
In case of branch, the breakpoint is set to both - next instruction
and possible branch address. This requires at least 2 breakpoints
supported in the CPU however this is a must for ARMv6/v7 CPUs.

Reviewed by: imp
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Obtained from: Semihalf
Sponsored by: Juniper Networks Inc.
Differential Revision: https://reviews.freebsd.org/D4037

Fix cp15 PAR definition and function. While here, add cp15 ATS1CPW
function which checks an address for privileged (PL1) write access.
The function is inlined so it does not bring any cost, but makes
function set for checking privileged access complete.

Approved by: kib (mentor)

Provide armv4/v5 implementations of several of the armv6 cache maintenance
functions. This will make it possible to use the same busdma code for all
arm platforms v4 thru v7.

Rename dcache_dma_preread() to dcache_inv_poc_dma() to make it clear that it
is a dcache invalidate to point of coherency just like dcache_inv_poc(), but
a slightly different version specific to dma operations. Elaborate the
comment about how and why it's different.

Use pmap_quick* functions in armv6 busdma, for bounce buffers and cache maintenance. This makes it safe to sync buffers that have no VA mapping associated with the busdma map, but may have other mappings, possibly on different CPUs. This also makes it safe to sync unmapped bounce buffers in non-sleepable thread contexts.

Similar to r286787 for x86, this treats userspace buffers the same as unmapped buffers and no longer borrows the UVA for sync operations.

Submitted by: Svatopluk Kraus <onwahe@gmail.com> (earlier revision)
Tested by: Svatopluk Kraus
Differential Revision: https://reviews.freebsd.org/D3869

Add more cp15_ functions, and use them in cpufunc.c where possible.

Add assertions that the addresses passed to tlb maintenance are page-aligned.

Perform cache writebacks and invalidations in the correct (inner to outer
or vice versa) order, and add comments that explain that.

Consistantly use 'va' as the variable name for virtual addresses.

Submitted by: Michal Meloun <meloun@miracle.cz>

cpu-v6.h should only be used in the kernel, add an error to enforce this.

Add new CP15 operations and DB_SHOW_COMMAND to print CP15 registers

Submitted by: Wojciech Macek <wma@semihalf.com>
Reviewed by: imp, Michal Meloun <meloun@miracle.cz>
Obtained from: Semihalf

Add the generic timer registers to sysreg.h and cpu-v6.h, and use the
access functions in the generic timer driver.

Differential Revision: https://reviews.freebsd.org/D2198
Sponsored by: The FreeBSD Foundation

Add minimum cache line sizes to struct cpuinfo, use them in the new cache
maintenance routines. Also add a routine to invalidate the branch cache.

Submitted by: Michal Meloun

Add the User and PL1 read only and reqd write thread ID registers.

Sponsored by: The FreeBSD Foundation

Add accessors for the ARM CP15 performance monitor registers. Also ensure
that some #ifdef SMP code is also conditional on __ARM_ARCH >= 7; we don't
support SMP on armv6, but some drivers and modules are compiled with it
forced on via the compiler command line.

Fix a "decl is not a prototype" error noticed by gcc (but not clang).

Add new TLB and cache maintainence functions for armv6 and armv7. These
are inline functions that handle all the routine maintenance operations
except the flush-all and invalidate-all routines which are required only
during early kernel init.

These inline functions should be very much faster than the old mechanism
that involved jumping through the big cpufuncs table, especially for
common operations such as invalidating a single TLB entry. Note that
nothing is calling these yet, this just is just required infrastructure
for upcoming changes to the pmap-v6 code.

Add new code to read and parse cpu identification data using the new CPUID
mechanism defined for armv7 (and also present on some armv6 chips including
the arm1176 used on rpi). The information is parsed into a global cpuinfo
structure, which will be used by (upcoming) new cache and tlb maintenance
code to handle cpu-specific variations of the maintence sequences.

Submitted by: Svatopluk Kraus <onwahe@gmail.com>,
Michal Meloun <meloun@miracle.cz