Revert r330897:

This was intended to be a non-functional change. It wasn't. The commit
message was thus wrong. In addition it broke arm, and merged crypto
related code.

Revert with prejudice.

This revert skips files touched in r316370 since that commit was since
MFCed. This revert also skips files that require $FreeBSD$ property
changes.

Thank you to those who helped me get out of this mess including but not
limited to gonzo, kevans, rgrimes.

Requested by: gjb (re)

MFC r317055,r317056 (glebius): Include sys/vmmeter.h as included

r317055: All these files need sys/vmmeter.h, but now they got it implicitly
included via sys/pcpu.h.

r317056: Typo!

Partial merge of the SPDX changes

These changes are incomplete but are making it difficult
to determine what other changes can/should be merged.

No objections from: pfg

Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle.
Prune svn:mergeinfo from the new branch, as nothing has been merged
here.

Additional commits post-branch will follow.

Approved by: re (implicit)
Sponsored by: The FreeBSD Foundation

[intrng] Migrate the intrng code from sys/arm/arm to sys/kern/subr_intr.c.

The ci20 port (by kan@) is going to reuse almost all of the intrng code
since the SoC in question looks suspiciously like someone took an ARM
SoC design and replaced the ARM core with a MIPS core.

* migrate out the code;
* rename ARM_ -> INTR_;
* rename arm_ -> intr_;
* move the interrupt flush routine from intr.c / intrng.c into
arm/machdep_intr.c - removing the code duplication and removing
the ARM specific bits from here.

Thanks to the Star Wars: The Force Awakens premiere line for allowing
me a couple hours of quiet time to finish the universe builds.

Tested:

* make universe

TODO:

* The structure definitions in subr_intr.c still includes machine/intr.h
which requires one duplicates all of the intrng definitions in
the platform code (which kan has done, and I think we don't have to.)

Instead I should break out the generic things (function declarations,
common intr structures, etc) into a separate header.

* Kan has requested I make the PIC based IPI stuff optional.

Remove trailing whitespace from sys/arm/arm

Fix the vmstat -i output on ARM.

The consumers of hw.intrnames expect a NULL byte at end of the string
containing the interrupt names.

On ARM all the interrupt name slots are initialized and this leave no room
for the terminating NULL byte, which makes vmstat read beyond the end of
intrnames.

PR: 199891
Tested on: RPi 2 and BeagleBone Black

Don't include unneeded files in the arm machine/fdt.h. While here, remove
it from more files.

Add ARMv7 performance monitoring counters.

Differential Revision: https://reviews.freebsd.org/D1687
Reviewed by: rpaulo
Sponsored by: DARPA, AFRL

Pull out the fdt mapping code into intr.c. The arm_intrng branch also
defines this function allowing the mapping method to change when we move
to it.

Use a SYSINIT to init the array of interrupt names on arm. This was
called from initarm() in arm/machdep.c, but many legacy arm platforms
have their own private initarm(), so a SYSINIT fixes everyone.

Reported by: jmg

Eliminate one of the causes of spurious interrupts on armv6. The arm weak
memory ordering model allows writes to different devices to complete out
of order, leading to a situation where the write that clears an interrupt
source at a device can complete after a write that unmasks and EOIs the
interrupt at the interrupt controller, leading to a spurious re-interrupt.

This adds a generic barrier function specific to the needs of interrupt
controllers, and calls that function from the GIC and TI AINTC controllers.
There may still be other soc-specific controllers that need to make the call.

Reviewed by: cognet, Svatopluk Kraus <onwahe@gmail.com>
MFC after: 3 days

Eliminate irq_dispatch.S. Move the data items it contained into arm/intr.c
and the functionality it provided into arm/exception.S. Rename the main
irq handling routine from arm_handler_execute() to arm_irq_handler() to
make it more congruent with how other exception handlers are named, and
also update its signature to reflect what has long been reality: it is
passed just a trapframe pointer, no interrupt number argument.

Add polarity and level support to ARM GIC

Add suport for setting triggering level and polarity in GIC.
New function pointer was added to nexus which corresponds
to the function which sets level/sense in the hardware (GIC).

Submitted by: Wojciech Macek <wma@semihalf.com>
Obtained from: Semihalf

Restore the irq number to the display string; I fumbled this in the previous
commit while trying to make the code internally self-consistant.

Approved by: cognet (mentor)
Obtained from: Christoph Mallon

Fix a buffer overrun while pre-formatting the names array, perpetrated in
the prior commit. Use essentially the same sprintf() statement for both
formatting and pre-formatting, and use a format string which eliminates the
need for an extra temporary buffer when formatting the name.

Noted by: Christoph Mallon
Pointy hat to: ian
Approved by: cognet (mentor)

Eliminate the need for an intermediate array of indices into the arrays of
interrupt counts and names, by making the names into an array of fixed-length
strings that can be directly indexed. This eliminates extra memory accesses
on every interrupt to increment the counts.

As a side effect, it also fixes a bug that would corrupt the names data
if a name was longer than MAXCOMLEN, which led to incorrect vmstat -i output.

Approved by: cognet (mentor)

Final whitespace trim.

Pass the previously returned IRQ back to arm_get_next_irq() so that
the implementation can guarantee forward progress in the event of
a stuck interrupt or interrupt storm. This is especially critical
for fast interrupt handlers, as they can cause a hard hang in that
case. When first called, arm_get_next_irq() is passed -1.

Obtained from: Juniper Networks, Inc.

ARM interrupts improvements.

- Fix nexus_setup_intr() abuse of setting up multiple IRQs in one go. Calling
arm_setup_irqhandler() in loop is bogus, as there's just one cookie given
from the caller and it is overwritten in each iteration so that only the
last handler's cookie value prevails.

- Proper intr masking/unmasking handling: the IRQ source is masked at PIC level
only after the last handler has been removed from the list.

Reviewed by: cognet, imp, sam, stass
Obtained from: Grzegorz Bernacki gjb ! semihalf dot com

Do not modify td->td_intr_nesting_level, it is now done in the MI code.
This fixes the cpu time being falsely reported as interrupt time.

MFC after: 3 days

On the AT91, we need to write on the EOI register after we handle an
interrupt. So, add a new function pointer, arm_post_filter, which defaults
to NULL, and which will be used as the post_filter arg for
intr_event_create(). Set it properly for the AT91, so that it boots again.

Reported by: hps

- Add the interrupt vector number to intr_event_create so MI code can
lookup hard interrupt events by number. Ignore the irq# for soft intrs.
- Add support to cpuset for binding hardware interrupts. This has the
side effect of binding any ithread associated with the hard interrupt.
As per restrictions imposed by MD code we can only bind interrupts to
a single cpu presently. Interrupts can be 'unbound' by binding them
to all cpus.

Reviewed by: jhb
Sponsored by: Nokia

Add a MI intr_event_handle() routine for the non-INTR_FILTER case. This
allows all the INTR_FILTER #ifdef's to be removed from the MD interrupt
code.
- Rename the intr_event 'eoi', 'disable', and 'enable' hooks to
'post_filter', 'pre_ithread', and 'post_ithread' to be less x86-centric.
Also, add a comment describe what the MI code expects them to do.
- On amd64, i386, and powerpc this is effectively a NOP.
- On arm, don't bother masking the interrupt unless the ithread is
scheduled in the non-INTR_FILTER case to match what INTR_FILTER did.
Also, don't bother unmasking the interrupt in the post_filter case if
we never masked it. The INTR_FILTER case had been doing this by having
arm_unmask_irq for the post_filter (formerly 'eoi') hook.
- On ia64, stray interrupts are now masked for the non-INTR_FILTER case.
They were already masked in the INTR_FILTER case.
- On sparc64, use the a NULL pre_ithread hook and use intr_enable_eoi() for
both the 'post_filter' and 'post_ithread' hooks to match what the
non-INTR_FILTER code did.
- On sun4v, retire the ithread wrapper hack by using an appropriate
'post_ithread' hook instead (it's what 'post_ithread'/'enable' was
designed to do even in 5.x).

Glanced at by: piso
Reviewed by: marius
Requested by: marius [1], [5]
Tested on: amd64, i386, arm, sparc64

Simplify the interrupt code a bit:
- Always include the ie_disable and ie_eoi methods in 'struct intr_event'
and collapse down to one intr_event_create() routine. The disable and
eoi hooks simply aren't used currently in the !INTR_FILTER case.
- Expand 'disab' to 'disable' in a few places.
- Use function casts for arm and i386:intr_eoi_src() instead of wrapper
routines since to trim one extra indirection.

Compiled on: {arm,amd64,i386,ia64,ppc,sparc64} x {FILTER, !FILTER}
Tested on: {amd64,i386} x {FILTER, !FILTER}

Add preliminary support for binding interrupts to CPUs:
- Add a new intr_event method ie_assign_cpu() that is invoked when the MI
code wishes to bind an interrupt source to an individual CPU. The MD
code may reject the binding with an error. If an assign_cpu function
is not provided, then the kernel assumes the platform does not support
binding interrupts to CPUs and fails all requests to do so.
- Bind ithreads to CPUs on their next execution loop once an interrupt
event is bound to a CPU. Only shared ithreads are bound. We currently
leave private ithreads for drivers using filters + ithreads in the
INTR_FILTER case unbound.
- A new intr_event_bind() routine is used to bind an interrupt event to
a CPU.
- Implement binding on amd64 and i386 by way of the existing pic_assign_cpu
PIC method.
- For x86, provide a 'intr_bind(IRQ, cpu)' wrapper routine that looks up
an interrupt source and binds its interrupt event to the specified CPU.
MI code can currently (ab)use this by doing:

intr_bind(rman_get_start(irq_res), cpu);

however, I plan to add a truly MI interface (probably a bus_bind_intr(9))
where the implementation in the x86 nexus(4) driver would end up calling
intr_bind() internally.

Requested by: kmacy, gallatin, jeff
Tested on: {amd64, i386} x {regular, INTR_FILTER}

INTR_FILTER bits for arm

Approved by: re (blanket)

Rework the PCPU_* (MD) interface:
- Rename PCPU_LAZY_INC into PCPU_INC
- Add the PCPU_ADD interface which just does an add on the pcpu member
given a specific value.

Note that for most architectures PCPU_INC and PCPU_ADD are not safe.
This is a point that needs some discussions/work in the next days.

Reviewed by: alc, bde
Approved by: jeff (mentor)

In some particular cases (like in pccard and pccbb), the real device
handler is wrapped in a couple of functions - a filter wrapper and an
ithread wrapper. In this case (and just in this case), the filter
wrapper could ask the system to schedule the ithread and mask the
interrupt source if the wrapped handler is composed of just an ithread
handler: modify the "old" interrupt code to make it support
this situation, while the "new" interrupt code is already ok.

Discussed with: jhb

o break newbus api: add a new argument of type driver_filter_t to
bus_setup_intr()

o add an int return code to all fast handlers

o retire INTR_FAST/IH_FAST

For more info: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=465712+0+current/freebsd-current

Reviewed by: many
Approved by: re@

Increate cnt.v_intr on interrupt.

Tweak how the MD code calls the fooclock() methods some. Instead of
passing a pointer to an opaque clockframe structure and requiring the
MD code to supply CLKF_FOO() macros to extract needed values out of the
opaque structure, just pass the needed values directly. In practice this
means passing the pair (usermode, pc) to hardclock() and profclock() and
passing the boolean (usermode) to hardclock_cpu() and hardclock_process().
Other details:
- Axe clockframe and CLKF_FOO() macros on all architectures. Basically,
all the archs were taking a trapframe and converting it into a clockframe
one way or another. Now they can just extract the PC and usermode values
directly out of the trapframe and pass it to fooclock().
- Renamed hardclock_process() to hardclock_cpu() as the latter is more
accurate.
- On Alpha, we now run profclock() at hz (profhz == hz) rather than at
the slower stathz.
- On Alpha, for the TurboLaser machines that don't have an 8254
timecounter, call hardclock() directly. This removes an extra
conditional check from every clock interrupt on Alpha on the BSP.
There is probably room for even further pruning here by changing Alpha
to use the simplified timecounter we use on x86 with the lapic timer
since we don't get interrupts from the 8254 on Alpha anyway.
- On x86, clkintr() shouldn't ever be called now unless using_lapic_timer
is false, so add a KASSERT() to that affect and remove a condition
to slightly optimize the non-lapic case.
- Change prototypeof arm_handler_execute() so that it's first arg is a
trapframe pointer rather than a void pointer for clarity.
- Use KCOUNT macro in profclock() to lookup the kernel profiling bucket.

Tested on: alpha, amd64, arm, i386, ia64, sparc64
Reviewed by: bde (mostly)

Reorganize the interrupt handling code a bit to make a few things cleaner
and increase flexibility to allow various different approaches to be tried
in the future.
- Split struct ithd up into two pieces. struct intr_event holds the list
of interrupt handlers associated with interrupt sources.
struct intr_thread contains the data relative to an interrupt thread.
Currently we still provide a 1:1 relationship of events to threads
with the exception that events only have an associated thread if there
is at least one threaded interrupt handler attached to the event. This
means that on x86 we no longer have 4 bazillion interrupt threads with
no handlers. It also means that interrupt events with only INTR_FAST
handlers no longer have an associated thread either.
- Renamed struct intrhand to struct intr_handler to follow the struct
intr_foo naming convention. This did require renaming the powerpc
MD struct intr_handler to struct ppc_intr_handler.
- INTR_FAST no longer implies INTR_EXCL on all architectures except for
powerpc. This means that multiple INTR_FAST handlers can attach to the
same interrupt and that INTR_FAST and non-INTR_FAST handlers can attach
to the same interrupt. Sharing INTR_FAST handlers may not always be
desirable, but having sio(4) and uhci(4) fight over an IRQ isn't fun
either. Drivers can always still use INTR_EXCL to ask for an interrupt
exclusively. The way this sharing works is that when an interrupt
comes in, all the INTR_FAST handlers are executed first, and if any
threaded handlers exist, the interrupt thread is scheduled afterwards.
This type of layout also makes it possible to investigate using interrupt
filters ala OS X where the filter determines whether or not its companion
threaded handler should run.
- Aside from the INTR_FAST changes above, the impact on MD interrupt code
is mostly just 's/ithread/intr_event/'.
- A new MI ddb command 'show intrs' walks the list of interrupt events
dumping their state. It also has a '/v' verbose switch which dumps
info about all of the handlers attached to each event.
- We currently don't destroy an interrupt thread when the last threaded
handler is removed because it would suck for things like ppbus(8)'s
braindead behavior. The code is present, though, it is just under
#if 0 for now.
- Move the code to actually execute the threaded handlers for an interrrupt
event into a separate function so that ithread_loop() becomes more
readable. Previously this code was all in the middle of ithread_loop()
and indented halfway across the screen.
- Made struct intr_thread private to kern_intr.c and replaced td_ithd
with a thread private flag TDP_ITHREAD.
- In statclock, check curthread against idlethread directly rather than
curthread's proc against idlethread's proc. (Not really related to intr
changes)

Tested on: alpha, amd64, i386, sparc64
Tested on: arm, ia64 (older version of patch by cognet and marcel)

Make sure the interrupt is masked before processing it, or bad things
can happen.

- MFp4: modify slightly the arm intr API, there's arm CPUs with more than 32
interrupts.
- Implement teardown methods where appropriate.

Don't enable interrupts in the dispatcher, there's no need to do so.

Start all license statements with /*-

Enable interrupts once the active ones have been masked.

Enable interrupts as soon as the pending interrupts have been masked.

Implement interrupt counting, so that vmstat -i work.

Add new functions to know which irqs are pending, and to mask and unmask
interrupts, as these are CPU specific.
If the interrupt handler is not marked as INTR_FAST, don't unmask the
interrupt until it as been serviced.

ithread_schedule() now only takes one argument.

Import FreeBSD/arm kernel bits.
It only supports sa1110 (on simics) right now, but xscale support should come
soon.
Some of the initial work has been provided by :
Stephane Potvin <sepotvin at videotron.ca>
Most of this comes from NetBSD.