cpu_emergency_disable() was only used by mvme88k and can be removed now.

This removes one of the SCHED_LOCK usages in arch.
OK miod@

Remove copystr(9), unless used internally by copy{in,out}str.

Simplfity setregs() by passing it the ps_strings and switching
sys_execve() to return EJUSTRETURN.

setregs() is the MD routine used by sys_execve() to set up the
thread's trapframe and PCB such that, on 'return' to userspace, it
has the register values defined by the ABI and otherwise zero. It
had to set the syscall retval[] values previously because the normal
syscall return path overwrites a couple registers with the retval[]
values. By instead returning EJUSTRETURN that and some complexity
with program-counter handling on m88k and sparc64 goes away.

Also, give setregs() add a 'struct ps_strings *arginfo' argument
so powerpc, powerpc64, and sh can directly get argc/argv/envp
values for registers instead of copyin()ing the one in userspace.

Improvements from miod@ and millert@
Testing assistance miod@, kettenis@, and aoyama@
ok miod@ kettenis@

Reduce the number of places where `p_priority' and `p_stat' are set.

This refactoring will help future scheduler locking, in particular to
shrink the SCHED_LOCK().

No intended behavior change.

ok visa@

More "explicitely" -> "explicitly" in various comments.

ok guenther@ tb@ deraadt@

Kill SPINLOCK_SPIN_HOOK, use CPU_BUSY_CYCLE() instead.

ok visa@, kettenis@

Put the address of the per-cpu clock and interrupt registers into the cpu_info
struct, rather than fetching them from arrays every time they are needed.

ok aoyama@

Tweak the special handling of pid 1 in setregs()...and document that
while it's not 100% correct, it's okay

tested by aoyama@

Apply consistency to forever loops with continue and NOTREACHED

Same thought from kettenis@, ok krw@ phessler@ millert@

no md code wants lockmgr locks, so no md code needs to include sys/lock.h

with and ok miod@

Implement rw_cas() with atomic_cas_ulong() now, instead of a dedicated
routine.

Remove redundant regdump() output.

In regdump(), print DAE registers when they are valid, regardless of the
exception type.

Format string fixes for m88k; remove -Wno-format from the m88k kernels.

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Put the kernel VBR page at the beginning of the kernel text, as done on
mvme88k, to be able to unmap the page at address zero in the kernel, and have
the vbr page read-only after being initialized.

Fix setregs() to match prototype. Exposed by recent _types.h change, found the
hard way by aoyama@

Initialize a teeny few more globals to reasonable values to allow for printf()
to work as soon as cn_tab is initialized. Allows us to correctly printf or
panic early on unrecognized systems. This used to be the case but got lost a
long time ago.

Constify struct cmmu.

Make sure new processes get a clean FPU configuration upon startup, by
resetting the appropriate register in setregs(). Similar in spirit to what
m68k does (restoring a null frame), and necessary since m88k doesn't do lazy
fpu switching.

Switch m88k ports to ELF.

Make rusage totals, itimers, and profile settings per-process instead
of per-rthread. Handling of per-thread tick and runtime counters
inspired by how FreeBSD does it.

ok kettenis@

The exception vector page on m88k systems has always been the same page as
the one used by the firmware, which (at least on mvme88k) is at address zero.

This is unfortunate, since this means that NULL pointer dereferences in the
kernel are not caught, and writes cause havoc.

This behaviour was necessary to be able to use the PROM system call interface
during early bootstrap, without having to disassemble the VBR page and
update branches - which use pc-relative displacement - if we were to use
a different VBR address.

On mvme88k, change this and actually set up two VBR pages: one, over the PROM
page (except for the system call vectors), and another one in the kernel
image (which will be mapped read-only). We'll run with the PROM page until
the end of autoconf, and then switch to the kernel one, and unmap all the
PROM below it.

As a bonus, the final kernel page can use optimized data access fault
handlers, which do not have to check for faults caused by badaddr(), since
badaddr() is only used during device probe on buses which do not support
hotplug.

There are a few infrastructure collateral damage on aviion and luna88k, but
these ports do not change their behaviour yet.

Rename max_cpus to ncpusfound and compute it regardless of option
MULTIPROCESSOR.

Generic softinterrupt support for m88k based platforms, adapted from arm
with different locking mechanism. 88110 soft ipi are replaced with an
ipi callback which is checked upon return from exception (it can not be kept
as a softintr, as the generic softinterrupt code doesn't have per-cpu
pending softintr queues).

Introduce atomic_clear_int() as an MD atomic operation to perform atomic
exchange with zero; use it in the soft interrupt code to make it simpler
and faster.

Fix stupid logic bug in rw_cas_m88k(), makes NFS much happier on SMP kernels.

atomic_{set,clear}bits_int were not safe enough on 88110 systems, as they
can be interrupted by NMI; move the SMP version of these routines from
inlines to a separate file (kernel text shrinks 20KB...).

Since the implementation for 88110 becomes really hairy, the pre-main() code
is responsible for copying the appropriate code over for kernels configured
for both 88100 and 88110 cpus, to avoid having to choose the atomicity
strategy at runtime. Hairy, I said.

This gets GENERIC.MP run much further on 197DP. Not enough to reach multiuser
mode, but boots up to starting sshd and then panics.

typo

More 88110 SMP work. Contains, horribly entangled:
- dma_cachectl() split into a ``local cpu only'' and ``all cpus'', and an ipi
to broadcast ``local dma_cachectl'' is added.
- cpu_info fields are rearranged, to have the 88100-specific information
and the 88110-specific information overlap, and has many more 88110
ugly things.
- more ipi handling in the 197-specific area. Since it is not possible to
have the second processor receive any hardware interrupt (selection
is done on a level basis via ISEL, and we definitely do not want the
main cpu to lose interrupts), the best we can do is to inflict ourselves
a soft interrupt for late ipi processing. It gets used for softclock and
hardclock on the secondary processor, but since the soft interrupt
dispatcher doesn't have an exception frame, we have to remember parts
of it to build a fake clockframe from the soft ipi handler (ugly but
works).

This now lets GENERIC.MP run a few userland binaries before bugs trigger.

Proper cpu_unidle() function for MP kernels. ok art@ long ago

On 88100 cpus, make sure userland processes start with SNIP and SFIP valid
(i.e. with the valid bit set in them). Found the hard way by Anders Gavare
trying his latest gxemul, proves the hardware is more permitting than one
would expect it to be...

Remove the last debug bit from the PSR on 88110: do not force memory accesses
instructions to be serialized (this defeats the purpose of having a superscalar
processor, and accesses to volatile variables are done with explicit memory
barriers anyway).

This brings a HUGE speedup: openssl speed -elapsed shows AES is 90% faster,
blowfish is 75% faster, and sha1 is 50% faster. Not so bad!

However, doing this increases the pressure on the processor bus, so it is
necessary to increase the processor bus timeout on 40MHz boards again (to 256
usec). ``black cat'' 50MHz boards seem to be unaffected, so they remain at
64 usec.

Move the cmmu lock back from 8820x-specific code to global, and use it on
MVME197DP to serialize 88410 operations.

I honestly do not remember what 88110 errata convinced me to disable
out-of-order (superscalar) execution on these processors.

Since OoO brings a nice 50% to 250% speedup (as shown by ``openssl speed''),
it is definitely worth enabling.

xmem can not be used as a reliable atomic operation, they way we do them,
so do the naive operations with interrupts disabled, and an interlock held
if MULTIPROCESSOR.

Improve setregs() so that it performs the equivalent of the first two
instructions of the new binary, which allows them to be skipped unconditionaly.

When setting up the vectors page, do not put a nop as the first instruction
if the processor is a 88110, since only the 88100 suffers from the ``will
fetch one instruction too far'' bug.

Provide a specific rw_cas() function for MP kernels, simulating a really
atomic compare-and-swap operation.

Move the cmmu lock to 88200-specific code. 88110 MP code will use ipis
and will not require such a lock.

Fix setregs() on 88110, so that binaries do not skip their first instruction.
Makes binaries beyond init(8) running on MVME197, to some extent.

Replace many ``unsigned'' variables with ``unsigned int'', ``u_int'' or other
appropriate types. No functional change.

Check for curcpu()->ci_softintr being nonzero before invoking dosoftint(),
in the common case, this saves us two potentially expensive setipl() calls.

Resolve the struct trapframe tf_mode/tf_flags duality by getting rid of
the tf_mode name. This field only contains flags used internally by the
exception processing code.

Merge the ci_alive and ci_primary boolean values of struct cpu_info into
a single ci_flags bitfield.

Also, set_cpu_number() will no longer set CIF_PRIMARY on the primary processor,
it's up to the initialization code to do this.

On MULTIPROCESSOR kernels, don't forget to grab the kernel lock when
processing soft interrupts; and there was much rejoicing.

Keep a pending software interrupts mask per processor, instead of having it
global; and only schedule software interrupts on the currently running cpu.

Remove the now unused idle_u, and call the secondary processors startup
stack a startup stack.

When a secondary cpu gets its interrupt pin stuck, be sure to savectx
and put the process it was running back on the run queue (unless this was
the idle proc).

Do not flag a processor as ``alive'' until it really is ready to accept IPIs.

Remove sir_lock, superseded by the atomic bit operations.

Make context switching much more MI:
- Move the functionality of choosing a process from cpu_switch into
a much simpler function: cpu_switchto. Instead of having the locore
code walk the run queues, let the MI code choose the process we
want to run and only implement the context switching itself in MD
code.
- Let MD context switching run without worrying about spls or locks.
- Instead of having the idle loop implemented with special contexts
in MD code, implement one idle proc for each cpu. make the idle
loop MI with MD hooks.
- Change the proc lists from the old style vax queues to TAILQs.
- Change the sleep queue from vax queues to TAILQs. This makes
wakeup() go from O(n^2) to O(n)

there will be some MD fallout, but it will be fixed shortly.
There's also a few cleanups to be done after this.

deraadt@, kettenis@ ok

Use atomic operations to operate on netisr, instead of clearing it at splhigh.
This changes nothing on legacy architectures, but is a bit faster (and simpler)
on the interesting ones.

Send an IPI in signotify() if the process runs on a different processor,
similar to the fix which went into i386 and amd64 a few weeks ago.

In spl0(), really process soft interrupts at IPL_SOFT instead of whatever
level we were at.

Use atomic operations to change the pending software interrupt mask.

The softintr lock needs to be an __mp_lock, not a __cpu_simple_lock.

Replace gazillions of badvaddr() or badwordaddr() calls with badaddr() calls.
With a few prototype declarations shuffling, this finally allows
<machine/locore.h> to die.

Clean the internal m88k trap type codes; while there, simplify and
factorize the build of the VBR page betweem luna88k and mvme88k.

Tested by aoyama@ and I.

nmihand() is mvme88k only, move it out of m88k common area.

Factorize soft interrupt code between luna88k and mvme88k.

Oops, simplelock leftover.

Replace simplelocks with __cpu_simple_locks for cmmu and pmap locking,
for the MULTIPROCESSOR case.

Switch m88k ports to __HAVE_CPUINFO. Current cpu pointer is held in SR0
on all running processors.
Tested aoyama@ and I

Maintain an interrupt depth counter for CLKF_INTR to be correct.
Tested aoyama@ martin@ and I.

Invoke regdump() on abort if compiled without DDB support.

Merge <machine/cpu_number.h> into <machine/cpu.h>, preparing for intrusive
changes.

Remove commented out dead wrong FPU handling code in setregs().

Stop mapping the u area at fixed UADDR in addition to its actual va.
While there, attempt to clean and comment stack usage in the kernel.
No functional change.

From the m88k SMP tree; help&test martin@

Change the size parameter of cmmu_flush_tlb() from bytes to pages. This makes
things easier for the callers, and allows us to inline the "fewer than 4 pages"
situation for speed.

Use [XNF]IP_ADDR constants rather than their numeric values.

Introduce sparc-like CPU_ISFOO macros, to short-circuit evaluations when
compiling for one flavour only.

This makes code slightly smaller on all m88k-based platforms (my roughly
1KB), and saves more than meets the eye on luna88k, which is m88100-based.

More code shuffling, probably the final touch.

Remove copystr(9), unless used internally by copy{in,out}str.

Simplfity setregs() by passing it the ps_strings and switching
sys_execve() to return EJUSTRETURN.

setregs() is the MD routine used by sys_execve() to set up the
thread's trapframe and PCB such that, on 'return' to userspace, it
has the register values defined by the ABI and otherwise zero. It
had to set the syscall retval[] values previously because the normal
syscall return path overwrites a couple registers with the retval[]
values. By instead returning EJUSTRETURN that and some complexity
with program-counter handling on m88k and sparc64 goes away.

Also, give setregs() add a 'struct ps_strings *arginfo' argument
so powerpc, powerpc64, and sh can directly get argc/argv/envp
values for registers instead of copyin()ing the one in userspace.

Improvements from miod@ and millert@
Testing assistance miod@, kettenis@, and aoyama@
ok miod@ kettenis@

Reduce the number of places where `p_priority' and `p_stat' are set.

This refactoring will help future scheduler locking, in particular to
shrink the SCHED_LOCK().

No intended behavior change.

ok visa@

More "explicitely" -> "explicitly" in various comments.

ok guenther@ tb@ deraadt@

Kill SPINLOCK_SPIN_HOOK, use CPU_BUSY_CYCLE() instead.

ok visa@, kettenis@

Put the address of the per-cpu clock and interrupt registers into the cpu_info
struct, rather than fetching them from arrays every time they are needed.

ok aoyama@

Tweak the special handling of pid 1 in setregs()...and document that
while it's not 100% correct, it's okay

tested by aoyama@

Apply consistency to forever loops with continue and NOTREACHED

Same thought from kettenis@, ok krw@ phessler@ millert@

no md code wants lockmgr locks, so no md code needs to include sys/lock.h

with and ok miod@

Implement rw_cas() with atomic_cas_ulong() now, instead of a dedicated
routine.

Remove redundant regdump() output.

In regdump(), print DAE registers when they are valid, regardless of the
exception type.

Format string fixes for m88k; remove -Wno-format from the m88k kernels.

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Put the kernel VBR page at the beginning of the kernel text, as done on
mvme88k, to be able to unmap the page at address zero in the kernel, and have
the vbr page read-only after being initialized.

Fix setregs() to match prototype. Exposed by recent _types.h change, found the
hard way by aoyama@

Initialize a teeny few more globals to reasonable values to allow for printf()
to work as soon as cn_tab is initialized. Allows us to correctly printf or
panic early on unrecognized systems. This used to be the case but got lost a
long time ago.

Constify struct cmmu.

Make sure new processes get a clean FPU configuration upon startup, by
resetting the appropriate register in setregs(). Similar in spirit to what
m68k does (restoring a null frame), and necessary since m88k doesn't do lazy
fpu switching.

Switch m88k ports to ELF.

Make rusage totals, itimers, and profile settings per-process instead
of per-rthread. Handling of per-thread tick and runtime counters
inspired by how FreeBSD does it.

ok kettenis@

The exception vector page on m88k systems has always been the same page as
the one used by the firmware, which (at least on mvme88k) is at address zero.

This is unfortunate, since this means that NULL pointer dereferences in the
kernel are not caught, and writes cause havoc.

This behaviour was necessary to be able to use the PROM system call interface
during early bootstrap, without having to disassemble the VBR page and
update branches - which use pc-relative displacement - if we were to use
a different VBR address.

On mvme88k, change this and actually set up two VBR pages: one, over the PROM
page (except for the system call vectors), and another one in the kernel
image (which will be mapped read-only). We'll run with the PROM page until
the end of autoconf, and then switch to the kernel one, and unmap all the
PROM below it.

As a bonus, the final kernel page can use optimized data access fault
handlers, which do not have to check for faults caused by badaddr(), since
badaddr() is only used during device probe on buses which do not support
hotplug.

There are a few infrastructure collateral damage on aviion and luna88k, but
these ports do not change their behaviour yet.

Rename max_cpus to ncpusfound and compute it regardless of option
MULTIPROCESSOR.

Generic softinterrupt support for m88k based platforms, adapted from arm
with different locking mechanism. 88110 soft ipi are replaced with an
ipi callback which is checked upon return from exception (it can not be kept
as a softintr, as the generic softinterrupt code doesn't have per-cpu
pending softintr queues).

Introduce atomic_clear_int() as an MD atomic operation to perform atomic
exchange with zero; use it in the soft interrupt code to make it simpler
and faster.

Fix stupid logic bug in rw_cas_m88k(), makes NFS much happier on SMP kernels.

atomic_{set,clear}bits_int were not safe enough on 88110 systems, as they
can be interrupted by NMI; move the SMP version of these routines from
inlines to a separate file (kernel text shrinks 20KB...).

Since the implementation for 88110 becomes really hairy, the pre-main() code
is responsible for copying the appropriate code over for kernels configured
for both 88100 and 88110 cpus, to avoid having to choose the atomicity
strategy at runtime. Hairy, I said.

This gets GENERIC.MP run much further on 197DP. Not enough to reach multiuser
mode, but boots up to starting sshd and then panics.

typo

More 88110 SMP work. Contains, horribly entangled:
- dma_cachectl() split into a ``local cpu only'' and ``all cpus'', and an ipi
to broadcast ``local dma_cachectl'' is added.
- cpu_info fields are rearranged, to have the 88100-specific information
and the 88110-specific information overlap, and has many more 88110
ugly things.
- more ipi handling in the 197-specific area. Since it is not possible to
have the second processor receive any hardware interrupt (selection
is done on a level basis via ISEL, and we definitely do not want the
main cpu to lose interrupts), the best we can do is to inflict ourselves
a soft interrupt for late ipi processing. It gets used for softclock and
hardclock on the secondary processor, but since the soft interrupt
dispatcher doesn't have an exception frame, we have to remember parts
of it to build a fake clockframe from the soft ipi handler (ugly but
works).

This now lets GENERIC.MP run a few userland binaries before bugs trigger.

Proper cpu_unidle() function for MP kernels. ok art@ long ago

On 88100 cpus, make sure userland processes start with SNIP and SFIP valid
(i.e. with the valid bit set in them). Found the hard way by Anders Gavare
trying his latest gxemul, proves the hardware is more permitting than one
would expect it to be...

Remove the last debug bit from the PSR on 88110: do not force memory accesses
instructions to be serialized (this defeats the purpose of having a superscalar
processor, and accesses to volatile variables are done with explicit memory
barriers anyway).

This brings a HUGE speedup: openssl speed -elapsed shows AES is 90% faster,
blowfish is 75% faster, and sha1 is 50% faster. Not so bad!

However, doing this increases the pressure on the processor bus, so it is
necessary to increase the processor bus timeout on 40MHz boards again (to 256
usec). ``black cat'' 50MHz boards seem to be unaffected, so they remain at
64 usec.

Move the cmmu lock back from 8820x-specific code to global, and use it on
MVME197DP to serialize 88410 operations.

I honestly do not remember what 88110 errata convinced me to disable
out-of-order (superscalar) execution on these processors.

Since OoO brings a nice 50% to 250% speedup (as shown by ``openssl speed''),
it is definitely worth enabling.

xmem can not be used as a reliable atomic operation, they way we do them,
so do the naive operations with interrupts disabled, and an interlock held
if MULTIPROCESSOR.

Improve setregs() so that it performs the equivalent of the first two
instructions of the new binary, which allows them to be skipped unconditionaly.

When setting up the vectors page, do not put a nop as the first instruction
if the processor is a 88110, since only the 88100 suffers from the ``will
fetch one instruction too far'' bug.

Provide a specific rw_cas() function for MP kernels, simulating a really
atomic compare-and-swap operation.

Move the cmmu lock to 88200-specific code. 88110 MP code will use ipis
and will not require such a lock.

Fix setregs() on 88110, so that binaries do not skip their first instruction.
Makes binaries beyond init(8) running on MVME197, to some extent.

Replace many ``unsigned'' variables with ``unsigned int'', ``u_int'' or other
appropriate types. No functional change.

Check for curcpu()->ci_softintr being nonzero before invoking dosoftint(),
in the common case, this saves us two potentially expensive setipl() calls.

Resolve the struct trapframe tf_mode/tf_flags duality by getting rid of
the tf_mode name. This field only contains flags used internally by the
exception processing code.

Merge the ci_alive and ci_primary boolean values of struct cpu_info into
a single ci_flags bitfield.

Also, set_cpu_number() will no longer set CIF_PRIMARY on the primary processor,
it's up to the initialization code to do this.

On MULTIPROCESSOR kernels, don't forget to grab the kernel lock when
processing soft interrupts; and there was much rejoicing.

Keep a pending software interrupts mask per processor, instead of having it
global; and only schedule software interrupts on the currently running cpu.

Remove the now unused idle_u, and call the secondary processors startup
stack a startup stack.

When a secondary cpu gets its interrupt pin stuck, be sure to savectx
and put the process it was running back on the run queue (unless this was
the idle proc).

Do not flag a processor as ``alive'' until it really is ready to accept IPIs.

Remove sir_lock, superseded by the atomic bit operations.

Make context switching much more MI:
- Move the functionality of choosing a process from cpu_switch into
a much simpler function: cpu_switchto. Instead of having the locore
code walk the run queues, let the MI code choose the process we
want to run and only implement the context switching itself in MD
code.
- Let MD context switching run without worrying about spls or locks.
- Instead of having the idle loop implemented with special contexts
in MD code, implement one idle proc for each cpu. make the idle
loop MI with MD hooks.
- Change the proc lists from the old style vax queues to TAILQs.
- Change the sleep queue from vax queues to TAILQs. This makes
wakeup() go from O(n^2) to O(n)

there will be some MD fallout, but it will be fixed shortly.
There's also a few cleanups to be done after this.

deraadt@, kettenis@ ok

Use atomic operations to operate on netisr, instead of clearing it at splhigh.
This changes nothing on legacy architectures, but is a bit faster (and simpler)
on the interesting ones.

Send an IPI in signotify() if the process runs on a different processor,
similar to the fix which went into i386 and amd64 a few weeks ago.

In spl0(), really process soft interrupts at IPL_SOFT instead of whatever
level we were at.

Use atomic operations to change the pending software interrupt mask.

The softintr lock needs to be an __mp_lock, not a __cpu_simple_lock.

Replace gazillions of badvaddr() or badwordaddr() calls with badaddr() calls.
With a few prototype declarations shuffling, this finally allows
<machine/locore.h> to die.

Clean the internal m88k trap type codes; while there, simplify and
factorize the build of the VBR page betweem luna88k and mvme88k.

Tested by aoyama@ and I.

nmihand() is mvme88k only, move it out of m88k common area.

Factorize soft interrupt code between luna88k and mvme88k.

Oops, simplelock leftover.

Replace simplelocks with __cpu_simple_locks for cmmu and pmap locking,
for the MULTIPROCESSOR case.

Switch m88k ports to __HAVE_CPUINFO. Current cpu pointer is held in SR0
on all running processors.
Tested aoyama@ and I

Maintain an interrupt depth counter for CLKF_INTR to be correct.
Tested aoyama@ martin@ and I.

Invoke regdump() on abort if compiled without DDB support.

Merge <machine/cpu_number.h> into <machine/cpu.h>, preparing for intrusive
changes.

Remove commented out dead wrong FPU handling code in setregs().

Stop mapping the u area at fixed UADDR in addition to its actual va.
While there, attempt to clean and comment stack usage in the kernel.
No functional change.

From the m88k SMP tree; help&test martin@

Change the size parameter of cmmu_flush_tlb() from bytes to pages. This makes
things easier for the callers, and allows us to inline the "fewer than 4 pages"
situation for speed.

Use [XNF]IP_ADDR constants rather than their numeric values.

Introduce sparc-like CPU_ISFOO macros, to short-circuit evaluations when
compiling for one flavour only.

This makes code slightly smaller on all m88k-based platforms (my roughly
1KB), and saves more than meets the eye on luna88k, which is m88100-based.

More code shuffling, probably the final touch.

Simplfity setregs() by passing it the ps_strings and switching
sys_execve() to return EJUSTRETURN.

setregs() is the MD routine used by sys_execve() to set up the
thread's trapframe and PCB such that, on 'return' to userspace, it
has the register values defined by the ABI and otherwise zero. It
had to set the syscall retval[] values previously because the normal
syscall return path overwrites a couple registers with the retval[]
values. By instead returning EJUSTRETURN that and some complexity
with program-counter handling on m88k and sparc64 goes away.

Also, give setregs() add a 'struct ps_strings *arginfo' argument
so powerpc, powerpc64, and sh can directly get argc/argv/envp
values for registers instead of copyin()ing the one in userspace.

Improvements from miod@ and millert@
Testing assistance miod@, kettenis@, and aoyama@
ok miod@ kettenis@

Reduce the number of places where `p_priority' and `p_stat' are set.

This refactoring will help future scheduler locking, in particular to
shrink the SCHED_LOCK().

No intended behavior change.

ok visa@

More "explicitely" -> "explicitly" in various comments.

ok guenther@ tb@ deraadt@

Kill SPINLOCK_SPIN_HOOK, use CPU_BUSY_CYCLE() instead.

ok visa@, kettenis@

Put the address of the per-cpu clock and interrupt registers into the cpu_info
struct, rather than fetching them from arrays every time they are needed.

ok aoyama@

Tweak the special handling of pid 1 in setregs()...and document that
while it's not 100% correct, it's okay

tested by aoyama@

Apply consistency to forever loops with continue and NOTREACHED

Same thought from kettenis@, ok krw@ phessler@ millert@

no md code wants lockmgr locks, so no md code needs to include sys/lock.h

with and ok miod@

Implement rw_cas() with atomic_cas_ulong() now, instead of a dedicated
routine.

Remove redundant regdump() output.

In regdump(), print DAE registers when they are valid, regardless of the
exception type.

Format string fixes for m88k; remove -Wno-format from the m88k kernels.

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Put the kernel VBR page at the beginning of the kernel text, as done on
mvme88k, to be able to unmap the page at address zero in the kernel, and have
the vbr page read-only after being initialized.

Fix setregs() to match prototype. Exposed by recent _types.h change, found the
hard way by aoyama@

Initialize a teeny few more globals to reasonable values to allow for printf()
to work as soon as cn_tab is initialized. Allows us to correctly printf or
panic early on unrecognized systems. This used to be the case but got lost a
long time ago.

Constify struct cmmu.

Make sure new processes get a clean FPU configuration upon startup, by
resetting the appropriate register in setregs(). Similar in spirit to what
m68k does (restoring a null frame), and necessary since m88k doesn't do lazy
fpu switching.

Switch m88k ports to ELF.

Make rusage totals, itimers, and profile settings per-process instead
of per-rthread. Handling of per-thread tick and runtime counters
inspired by how FreeBSD does it.

ok kettenis@

The exception vector page on m88k systems has always been the same page as
the one used by the firmware, which (at least on mvme88k) is at address zero.

This is unfortunate, since this means that NULL pointer dereferences in the
kernel are not caught, and writes cause havoc.

This behaviour was necessary to be able to use the PROM system call interface
during early bootstrap, without having to disassemble the VBR page and
update branches - which use pc-relative displacement - if we were to use
a different VBR address.

On mvme88k, change this and actually set up two VBR pages: one, over the PROM
page (except for the system call vectors), and another one in the kernel
image (which will be mapped read-only). We'll run with the PROM page until
the end of autoconf, and then switch to the kernel one, and unmap all the
PROM below it.

As a bonus, the final kernel page can use optimized data access fault
handlers, which do not have to check for faults caused by badaddr(), since
badaddr() is only used during device probe on buses which do not support
hotplug.

There are a few infrastructure collateral damage on aviion and luna88k, but
these ports do not change their behaviour yet.

Rename max_cpus to ncpusfound and compute it regardless of option
MULTIPROCESSOR.

Generic softinterrupt support for m88k based platforms, adapted from arm
with different locking mechanism. 88110 soft ipi are replaced with an
ipi callback which is checked upon return from exception (it can not be kept
as a softintr, as the generic softinterrupt code doesn't have per-cpu
pending softintr queues).

Introduce atomic_clear_int() as an MD atomic operation to perform atomic
exchange with zero; use it in the soft interrupt code to make it simpler
and faster.

Fix stupid logic bug in rw_cas_m88k(), makes NFS much happier on SMP kernels.

atomic_{set,clear}bits_int were not safe enough on 88110 systems, as they
can be interrupted by NMI; move the SMP version of these routines from
inlines to a separate file (kernel text shrinks 20KB...).

Since the implementation for 88110 becomes really hairy, the pre-main() code
is responsible for copying the appropriate code over for kernels configured
for both 88100 and 88110 cpus, to avoid having to choose the atomicity
strategy at runtime. Hairy, I said.

This gets GENERIC.MP run much further on 197DP. Not enough to reach multiuser
mode, but boots up to starting sshd and then panics.

typo

More 88110 SMP work. Contains, horribly entangled:
- dma_cachectl() split into a ``local cpu only'' and ``all cpus'', and an ipi
to broadcast ``local dma_cachectl'' is added.
- cpu_info fields are rearranged, to have the 88100-specific information
and the 88110-specific information overlap, and has many more 88110
ugly things.
- more ipi handling in the 197-specific area. Since it is not possible to
have the second processor receive any hardware interrupt (selection
is done on a level basis via ISEL, and we definitely do not want the
main cpu to lose interrupts), the best we can do is to inflict ourselves
a soft interrupt for late ipi processing. It gets used for softclock and
hardclock on the secondary processor, but since the soft interrupt
dispatcher doesn't have an exception frame, we have to remember parts
of it to build a fake clockframe from the soft ipi handler (ugly but
works).

This now lets GENERIC.MP run a few userland binaries before bugs trigger.

Proper cpu_unidle() function for MP kernels. ok art@ long ago

On 88100 cpus, make sure userland processes start with SNIP and SFIP valid
(i.e. with the valid bit set in them). Found the hard way by Anders Gavare
trying his latest gxemul, proves the hardware is more permitting than one
would expect it to be...

Remove the last debug bit from the PSR on 88110: do not force memory accesses
instructions to be serialized (this defeats the purpose of having a superscalar
processor, and accesses to volatile variables are done with explicit memory
barriers anyway).

This brings a HUGE speedup: openssl speed -elapsed shows AES is 90% faster,
blowfish is 75% faster, and sha1 is 50% faster. Not so bad!

However, doing this increases the pressure on the processor bus, so it is
necessary to increase the processor bus timeout on 40MHz boards again (to 256
usec). ``black cat'' 50MHz boards seem to be unaffected, so they remain at
64 usec.

Move the cmmu lock back from 8820x-specific code to global, and use it on
MVME197DP to serialize 88410 operations.

I honestly do not remember what 88110 errata convinced me to disable
out-of-order (superscalar) execution on these processors.

Since OoO brings a nice 50% to 250% speedup (as shown by ``openssl speed''),
it is definitely worth enabling.

xmem can not be used as a reliable atomic operation, they way we do them,
so do the naive operations with interrupts disabled, and an interlock held
if MULTIPROCESSOR.

Improve setregs() so that it performs the equivalent of the first two
instructions of the new binary, which allows them to be skipped unconditionaly.

When setting up the vectors page, do not put a nop as the first instruction
if the processor is a 88110, since only the 88100 suffers from the ``will
fetch one instruction too far'' bug.

Provide a specific rw_cas() function for MP kernels, simulating a really
atomic compare-and-swap operation.

Move the cmmu lock to 88200-specific code. 88110 MP code will use ipis
and will not require such a lock.

Fix setregs() on 88110, so that binaries do not skip their first instruction.
Makes binaries beyond init(8) running on MVME197, to some extent.

Replace many ``unsigned'' variables with ``unsigned int'', ``u_int'' or other
appropriate types. No functional change.

Check for curcpu()->ci_softintr being nonzero before invoking dosoftint(),
in the common case, this saves us two potentially expensive setipl() calls.

Resolve the struct trapframe tf_mode/tf_flags duality by getting rid of
the tf_mode name. This field only contains flags used internally by the
exception processing code.

Merge the ci_alive and ci_primary boolean values of struct cpu_info into
a single ci_flags bitfield.

Also, set_cpu_number() will no longer set CIF_PRIMARY on the primary processor,
it's up to the initialization code to do this.

On MULTIPROCESSOR kernels, don't forget to grab the kernel lock when
processing soft interrupts; and there was much rejoicing.

Keep a pending software interrupts mask per processor, instead of having it
global; and only schedule software interrupts on the currently running cpu.

Remove the now unused idle_u, and call the secondary processors startup
stack a startup stack.

When a secondary cpu gets its interrupt pin stuck, be sure to savectx
and put the process it was running back on the run queue (unless this was
the idle proc).

Do not flag a processor as ``alive'' until it really is ready to accept IPIs.

Remove sir_lock, superseded by the atomic bit operations.

Make context switching much more MI:
- Move the functionality of choosing a process from cpu_switch into
a much simpler function: cpu_switchto. Instead of having the locore
code walk the run queues, let the MI code choose the process we
want to run and only implement the context switching itself in MD
code.
- Let MD context switching run without worrying about spls or locks.
- Instead of having the idle loop implemented with special contexts
in MD code, implement one idle proc for each cpu. make the idle
loop MI with MD hooks.
- Change the proc lists from the old style vax queues to TAILQs.
- Change the sleep queue from vax queues to TAILQs. This makes
wakeup() go from O(n^2) to O(n)

there will be some MD fallout, but it will be fixed shortly.
There's also a few cleanups to be done after this.

deraadt@, kettenis@ ok

Use atomic operations to operate on netisr, instead of clearing it at splhigh.
This changes nothing on legacy architectures, but is a bit faster (and simpler)
on the interesting ones.

Send an IPI in signotify() if the process runs on a different processor,
similar to the fix which went into i386 and amd64 a few weeks ago.

In spl0(), really process soft interrupts at IPL_SOFT instead of whatever
level we were at.

Use atomic operations to change the pending software interrupt mask.

The softintr lock needs to be an __mp_lock, not a __cpu_simple_lock.

Replace gazillions of badvaddr() or badwordaddr() calls with badaddr() calls.
With a few prototype declarations shuffling, this finally allows
<machine/locore.h> to die.

Clean the internal m88k trap type codes; while there, simplify and
factorize the build of the VBR page betweem luna88k and mvme88k.

Tested by aoyama@ and I.

nmihand() is mvme88k only, move it out of m88k common area.

Factorize soft interrupt code between luna88k and mvme88k.

Oops, simplelock leftover.

Replace simplelocks with __cpu_simple_locks for cmmu and pmap locking,
for the MULTIPROCESSOR case.

Switch m88k ports to __HAVE_CPUINFO. Current cpu pointer is held in SR0
on all running processors.
Tested aoyama@ and I

Maintain an interrupt depth counter for CLKF_INTR to be correct.
Tested aoyama@ martin@ and I.

Invoke regdump() on abort if compiled without DDB support.

Merge <machine/cpu_number.h> into <machine/cpu.h>, preparing for intrusive
changes.

Remove commented out dead wrong FPU handling code in setregs().

Stop mapping the u area at fixed UADDR in addition to its actual va.
While there, attempt to clean and comment stack usage in the kernel.
No functional change.

From the m88k SMP tree; help&test martin@

Change the size parameter of cmmu_flush_tlb() from bytes to pages. This makes
things easier for the callers, and allows us to inline the "fewer than 4 pages"
situation for speed.

Use [XNF]IP_ADDR constants rather than their numeric values.

Introduce sparc-like CPU_ISFOO macros, to short-circuit evaluations when
compiling for one flavour only.

This makes code slightly smaller on all m88k-based platforms (my roughly
1KB), and saves more than meets the eye on luna88k, which is m88100-based.

More code shuffling, probably the final touch.

Reduce the number of places where `p_priority' and `p_stat' are set.

This refactoring will help future scheduler locking, in particular to
shrink the SCHED_LOCK().

No intended behavior change.

ok visa@

More "explicitely" -> "explicitly" in various comments.

ok guenther@ tb@ deraadt@

Kill SPINLOCK_SPIN_HOOK, use CPU_BUSY_CYCLE() instead.

ok visa@, kettenis@

Put the address of the per-cpu clock and interrupt registers into the cpu_info
struct, rather than fetching them from arrays every time they are needed.

ok aoyama@

Tweak the special handling of pid 1 in setregs()...and document that
while it's not 100% correct, it's okay

tested by aoyama@

Apply consistency to forever loops with continue and NOTREACHED

Same thought from kettenis@, ok krw@ phessler@ millert@

no md code wants lockmgr locks, so no md code needs to include sys/lock.h

with and ok miod@

Implement rw_cas() with atomic_cas_ulong() now, instead of a dedicated
routine.

Remove redundant regdump() output.

In regdump(), print DAE registers when they are valid, regardless of the
exception type.

Format string fixes for m88k; remove -Wno-format from the m88k kernels.

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Put the kernel VBR page at the beginning of the kernel text, as done on
mvme88k, to be able to unmap the page at address zero in the kernel, and have
the vbr page read-only after being initialized.

Fix setregs() to match prototype. Exposed by recent _types.h change, found the
hard way by aoyama@

Initialize a teeny few more globals to reasonable values to allow for printf()
to work as soon as cn_tab is initialized. Allows us to correctly printf or
panic early on unrecognized systems. This used to be the case but got lost a
long time ago.

Constify struct cmmu.

Make sure new processes get a clean FPU configuration upon startup, by
resetting the appropriate register in setregs(). Similar in spirit to what
m68k does (restoring a null frame), and necessary since m88k doesn't do lazy
fpu switching.

Switch m88k ports to ELF.

Make rusage totals, itimers, and profile settings per-process instead
of per-rthread. Handling of per-thread tick and runtime counters
inspired by how FreeBSD does it.

ok kettenis@

The exception vector page on m88k systems has always been the same page as
the one used by the firmware, which (at least on mvme88k) is at address zero.

This is unfortunate, since this means that NULL pointer dereferences in the
kernel are not caught, and writes cause havoc.

This behaviour was necessary to be able to use the PROM system call interface
during early bootstrap, without having to disassemble the VBR page and
update branches - which use pc-relative displacement - if we were to use
a different VBR address.

On mvme88k, change this and actually set up two VBR pages: one, over the PROM
page (except for the system call vectors), and another one in the kernel
image (which will be mapped read-only). We'll run with the PROM page until
the end of autoconf, and then switch to the kernel one, and unmap all the
PROM below it.

As a bonus, the final kernel page can use optimized data access fault
handlers, which do not have to check for faults caused by badaddr(), since
badaddr() is only used during device probe on buses which do not support
hotplug.

There are a few infrastructure collateral damage on aviion and luna88k, but
these ports do not change their behaviour yet.

Rename max_cpus to ncpusfound and compute it regardless of option
MULTIPROCESSOR.

Generic softinterrupt support for m88k based platforms, adapted from arm
with different locking mechanism. 88110 soft ipi are replaced with an
ipi callback which is checked upon return from exception (it can not be kept
as a softintr, as the generic softinterrupt code doesn't have per-cpu
pending softintr queues).

Introduce atomic_clear_int() as an MD atomic operation to perform atomic
exchange with zero; use it in the soft interrupt code to make it simpler
and faster.

Fix stupid logic bug in rw_cas_m88k(), makes NFS much happier on SMP kernels.

atomic_{set,clear}bits_int were not safe enough on 88110 systems, as they
can be interrupted by NMI; move the SMP version of these routines from
inlines to a separate file (kernel text shrinks 20KB...).

Since the implementation for 88110 becomes really hairy, the pre-main() code
is responsible for copying the appropriate code over for kernels configured
for both 88100 and 88110 cpus, to avoid having to choose the atomicity
strategy at runtime. Hairy, I said.

This gets GENERIC.MP run much further on 197DP. Not enough to reach multiuser
mode, but boots up to starting sshd and then panics.

typo

More 88110 SMP work. Contains, horribly entangled:
- dma_cachectl() split into a ``local cpu only'' and ``all cpus'', and an ipi
to broadcast ``local dma_cachectl'' is added.
- cpu_info fields are rearranged, to have the 88100-specific information
and the 88110-specific information overlap, and has many more 88110
ugly things.
- more ipi handling in the 197-specific area. Since it is not possible to
have the second processor receive any hardware interrupt (selection
is done on a level basis via ISEL, and we definitely do not want the
main cpu to lose interrupts), the best we can do is to inflict ourselves
a soft interrupt for late ipi processing. It gets used for softclock and
hardclock on the secondary processor, but since the soft interrupt
dispatcher doesn't have an exception frame, we have to remember parts
of it to build a fake clockframe from the soft ipi handler (ugly but
works).

This now lets GENERIC.MP run a few userland binaries before bugs trigger.

Proper cpu_unidle() function for MP kernels. ok art@ long ago

On 88100 cpus, make sure userland processes start with SNIP and SFIP valid
(i.e. with the valid bit set in them). Found the hard way by Anders Gavare
trying his latest gxemul, proves the hardware is more permitting than one
would expect it to be...

Remove the last debug bit from the PSR on 88110: do not force memory accesses
instructions to be serialized (this defeats the purpose of having a superscalar
processor, and accesses to volatile variables are done with explicit memory
barriers anyway).

This brings a HUGE speedup: openssl speed -elapsed shows AES is 90% faster,
blowfish is 75% faster, and sha1 is 50% faster. Not so bad!

However, doing this increases the pressure on the processor bus, so it is
necessary to increase the processor bus timeout on 40MHz boards again (to 256
usec). ``black cat'' 50MHz boards seem to be unaffected, so they remain at
64 usec.

Move the cmmu lock back from 8820x-specific code to global, and use it on
MVME197DP to serialize 88410 operations.

I honestly do not remember what 88110 errata convinced me to disable
out-of-order (superscalar) execution on these processors.

Since OoO brings a nice 50% to 250% speedup (as shown by ``openssl speed''),
it is definitely worth enabling.

xmem can not be used as a reliable atomic operation, they way we do them,
so do the naive operations with interrupts disabled, and an interlock held
if MULTIPROCESSOR.

Improve setregs() so that it performs the equivalent of the first two
instructions of the new binary, which allows them to be skipped unconditionaly.

When setting up the vectors page, do not put a nop as the first instruction
if the processor is a 88110, since only the 88100 suffers from the ``will
fetch one instruction too far'' bug.

Provide a specific rw_cas() function for MP kernels, simulating a really
atomic compare-and-swap operation.

Move the cmmu lock to 88200-specific code. 88110 MP code will use ipis
and will not require such a lock.

Fix setregs() on 88110, so that binaries do not skip their first instruction.
Makes binaries beyond init(8) running on MVME197, to some extent.

Replace many ``unsigned'' variables with ``unsigned int'', ``u_int'' or other
appropriate types. No functional change.

Check for curcpu()->ci_softintr being nonzero before invoking dosoftint(),
in the common case, this saves us two potentially expensive setipl() calls.

Resolve the struct trapframe tf_mode/tf_flags duality by getting rid of
the tf_mode name. This field only contains flags used internally by the
exception processing code.

Merge the ci_alive and ci_primary boolean values of struct cpu_info into
a single ci_flags bitfield.

Also, set_cpu_number() will no longer set CIF_PRIMARY on the primary processor,
it's up to the initialization code to do this.

On MULTIPROCESSOR kernels, don't forget to grab the kernel lock when
processing soft interrupts; and there was much rejoicing.

Keep a pending software interrupts mask per processor, instead of having it
global; and only schedule software interrupts on the currently running cpu.

Remove the now unused idle_u, and call the secondary processors startup
stack a startup stack.

When a secondary cpu gets its interrupt pin stuck, be sure to savectx
and put the process it was running back on the run queue (unless this was
the idle proc).

Do not flag a processor as ``alive'' until it really is ready to accept IPIs.

Remove sir_lock, superseded by the atomic bit operations.

Make context switching much more MI:
- Move the functionality of choosing a process from cpu_switch into
a much simpler function: cpu_switchto. Instead of having the locore
code walk the run queues, let the MI code choose the process we
want to run and only implement the context switching itself in MD
code.
- Let MD context switching run without worrying about spls or locks.
- Instead of having the idle loop implemented with special contexts
in MD code, implement one idle proc for each cpu. make the idle
loop MI with MD hooks.
- Change the proc lists from the old style vax queues to TAILQs.
- Change the sleep queue from vax queues to TAILQs. This makes
wakeup() go from O(n^2) to O(n)

there will be some MD fallout, but it will be fixed shortly.
There's also a few cleanups to be done after this.

deraadt@, kettenis@ ok

Use atomic operations to operate on netisr, instead of clearing it at splhigh.
This changes nothing on legacy architectures, but is a bit faster (and simpler)
on the interesting ones.

Send an IPI in signotify() if the process runs on a different processor,
similar to the fix which went into i386 and amd64 a few weeks ago.

In spl0(), really process soft interrupts at IPL_SOFT instead of whatever
level we were at.

Use atomic operations to change the pending software interrupt mask.

The softintr lock needs to be an __mp_lock, not a __cpu_simple_lock.

Replace gazillions of badvaddr() or badwordaddr() calls with badaddr() calls.
With a few prototype declarations shuffling, this finally allows
<machine/locore.h> to die.

Clean the internal m88k trap type codes; while there, simplify and
factorize the build of the VBR page betweem luna88k and mvme88k.

Tested by aoyama@ and I.

nmihand() is mvme88k only, move it out of m88k common area.

Factorize soft interrupt code between luna88k and mvme88k.

Oops, simplelock leftover.

Replace simplelocks with __cpu_simple_locks for cmmu and pmap locking,
for the MULTIPROCESSOR case.

Switch m88k ports to __HAVE_CPUINFO. Current cpu pointer is held in SR0
on all running processors.
Tested aoyama@ and I

Maintain an interrupt depth counter for CLKF_INTR to be correct.
Tested aoyama@ martin@ and I.

Invoke regdump() on abort if compiled without DDB support.

Merge <machine/cpu_number.h> into <machine/cpu.h>, preparing for intrusive
changes.

Remove commented out dead wrong FPU handling code in setregs().

Stop mapping the u area at fixed UADDR in addition to its actual va.
While there, attempt to clean and comment stack usage in the kernel.
No functional change.

From the m88k SMP tree; help&test martin@

Change the size parameter of cmmu_flush_tlb() from bytes to pages. This makes
things easier for the callers, and allows us to inline the "fewer than 4 pages"
situation for speed.

Use [XNF]IP_ADDR constants rather than their numeric values.

Introduce sparc-like CPU_ISFOO macros, to short-circuit evaluations when
compiling for one flavour only.

This makes code slightly smaller on all m88k-based platforms (my roughly
1KB), and saves more than meets the eye on luna88k, which is m88100-based.

More code shuffling, probably the final touch.

More "explicitely" -> "explicitly" in various comments.

ok guenther@ tb@ deraadt@

Kill SPINLOCK_SPIN_HOOK, use CPU_BUSY_CYCLE() instead.

ok visa@, kettenis@

Put the address of the per-cpu clock and interrupt registers into the cpu_info
struct, rather than fetching them from arrays every time they are needed.

ok aoyama@

Tweak the special handling of pid 1 in setregs()...and document that
while it's not 100% correct, it's okay

tested by aoyama@

Apply consistency to forever loops with continue and NOTREACHED

Same thought from kettenis@, ok krw@ phessler@ millert@

no md code wants lockmgr locks, so no md code needs to include sys/lock.h

with and ok miod@

Implement rw_cas() with atomic_cas_ulong() now, instead of a dedicated
routine.

Remove redundant regdump() output.

In regdump(), print DAE registers when they are valid, regardless of the
exception type.

Format string fixes for m88k; remove -Wno-format from the m88k kernels.

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Put the kernel VBR page at the beginning of the kernel text, as done on
mvme88k, to be able to unmap the page at address zero in the kernel, and have
the vbr page read-only after being initialized.

Fix setregs() to match prototype. Exposed by recent _types.h change, found the
hard way by aoyama@

Initialize a teeny few more globals to reasonable values to allow for printf()
to work as soon as cn_tab is initialized. Allows us to correctly printf or
panic early on unrecognized systems. This used to be the case but got lost a
long time ago.

Constify struct cmmu.

Make sure new processes get a clean FPU configuration upon startup, by
resetting the appropriate register in setregs(). Similar in spirit to what
m68k does (restoring a null frame), and necessary since m88k doesn't do lazy
fpu switching.

Switch m88k ports to ELF.

Make rusage totals, itimers, and profile settings per-process instead
of per-rthread. Handling of per-thread tick and runtime counters
inspired by how FreeBSD does it.

ok kettenis@

The exception vector page on m88k systems has always been the same page as
the one used by the firmware, which (at least on mvme88k) is at address zero.

This is unfortunate, since this means that NULL pointer dereferences in the
kernel are not caught, and writes cause havoc.

This behaviour was necessary to be able to use the PROM system call interface
during early bootstrap, without having to disassemble the VBR page and
update branches - which use pc-relative displacement - if we were to use
a different VBR address.

On mvme88k, change this and actually set up two VBR pages: one, over the PROM
page (except for the system call vectors), and another one in the kernel
image (which will be mapped read-only). We'll run with the PROM page until
the end of autoconf, and then switch to the kernel one, and unmap all the
PROM below it.

As a bonus, the final kernel page can use optimized data access fault
handlers, which do not have to check for faults caused by badaddr(), since
badaddr() is only used during device probe on buses which do not support
hotplug.

There are a few infrastructure collateral damage on aviion and luna88k, but
these ports do not change their behaviour yet.

Rename max_cpus to ncpusfound and compute it regardless of option
MULTIPROCESSOR.

Generic softinterrupt support for m88k based platforms, adapted from arm
with different locking mechanism. 88110 soft ipi are replaced with an
ipi callback which is checked upon return from exception (it can not be kept
as a softintr, as the generic softinterrupt code doesn't have per-cpu
pending softintr queues).

Introduce atomic_clear_int() as an MD atomic operation to perform atomic
exchange with zero; use it in the soft interrupt code to make it simpler
and faster.

Fix stupid logic bug in rw_cas_m88k(), makes NFS much happier on SMP kernels.

atomic_{set,clear}bits_int were not safe enough on 88110 systems, as they
can be interrupted by NMI; move the SMP version of these routines from
inlines to a separate file (kernel text shrinks 20KB...).

Since the implementation for 88110 becomes really hairy, the pre-main() code
is responsible for copying the appropriate code over for kernels configured
for both 88100 and 88110 cpus, to avoid having to choose the atomicity
strategy at runtime. Hairy, I said.

This gets GENERIC.MP run much further on 197DP. Not enough to reach multiuser
mode, but boots up to starting sshd and then panics.

typo

More 88110 SMP work. Contains, horribly entangled:
- dma_cachectl() split into a ``local cpu only'' and ``all cpus'', and an ipi
to broadcast ``local dma_cachectl'' is added.
- cpu_info fields are rearranged, to have the 88100-specific information
and the 88110-specific information overlap, and has many more 88110
ugly things.
- more ipi handling in the 197-specific area. Since it is not possible to
have the second processor receive any hardware interrupt (selection
is done on a level basis via ISEL, and we definitely do not want the
main cpu to lose interrupts), the best we can do is to inflict ourselves
a soft interrupt for late ipi processing. It gets used for softclock and
hardclock on the secondary processor, but since the soft interrupt
dispatcher doesn't have an exception frame, we have to remember parts
of it to build a fake clockframe from the soft ipi handler (ugly but
works).

This now lets GENERIC.MP run a few userland binaries before bugs trigger.

Proper cpu_unidle() function for MP kernels. ok art@ long ago

On 88100 cpus, make sure userland processes start with SNIP and SFIP valid
(i.e. with the valid bit set in them). Found the hard way by Anders Gavare
trying his latest gxemul, proves the hardware is more permitting than one
would expect it to be...

Remove the last debug bit from the PSR on 88110: do not force memory accesses
instructions to be serialized (this defeats the purpose of having a superscalar
processor, and accesses to volatile variables are done with explicit memory
barriers anyway).

This brings a HUGE speedup: openssl speed -elapsed shows AES is 90% faster,
blowfish is 75% faster, and sha1 is 50% faster. Not so bad!

However, doing this increases the pressure on the processor bus, so it is
necessary to increase the processor bus timeout on 40MHz boards again (to 256
usec). ``black cat'' 50MHz boards seem to be unaffected, so they remain at
64 usec.

Move the cmmu lock back from 8820x-specific code to global, and use it on
MVME197DP to serialize 88410 operations.

I honestly do not remember what 88110 errata convinced me to disable
out-of-order (superscalar) execution on these processors.

Since OoO brings a nice 50% to 250% speedup (as shown by ``openssl speed''),
it is definitely worth enabling.

xmem can not be used as a reliable atomic operation, they way we do them,
so do the naive operations with interrupts disabled, and an interlock held
if MULTIPROCESSOR.

Improve setregs() so that it performs the equivalent of the first two
instructions of the new binary, which allows them to be skipped unconditionaly.

When setting up the vectors page, do not put a nop as the first instruction
if the processor is a 88110, since only the 88100 suffers from the ``will
fetch one instruction too far'' bug.

Provide a specific rw_cas() function for MP kernels, simulating a really
atomic compare-and-swap operation.

Move the cmmu lock to 88200-specific code. 88110 MP code will use ipis
and will not require such a lock.

Fix setregs() on 88110, so that binaries do not skip their first instruction.
Makes binaries beyond init(8) running on MVME197, to some extent.

Replace many ``unsigned'' variables with ``unsigned int'', ``u_int'' or other
appropriate types. No functional change.

Check for curcpu()->ci_softintr being nonzero before invoking dosoftint(),
in the common case, this saves us two potentially expensive setipl() calls.

Resolve the struct trapframe tf_mode/tf_flags duality by getting rid of
the tf_mode name. This field only contains flags used internally by the
exception processing code.

Merge the ci_alive and ci_primary boolean values of struct cpu_info into
a single ci_flags bitfield.

Also, set_cpu_number() will no longer set CIF_PRIMARY on the primary processor,
it's up to the initialization code to do this.

On MULTIPROCESSOR kernels, don't forget to grab the kernel lock when
processing soft interrupts; and there was much rejoicing.

Keep a pending software interrupts mask per processor, instead of having it
global; and only schedule software interrupts on the currently running cpu.

Remove the now unused idle_u, and call the secondary processors startup
stack a startup stack.

When a secondary cpu gets its interrupt pin stuck, be sure to savectx
and put the process it was running back on the run queue (unless this was
the idle proc).

Do not flag a processor as ``alive'' until it really is ready to accept IPIs.

Remove sir_lock, superseded by the atomic bit operations.

Make context switching much more MI:
- Move the functionality of choosing a process from cpu_switch into
a much simpler function: cpu_switchto. Instead of having the locore
code walk the run queues, let the MI code choose the process we
want to run and only implement the context switching itself in MD
code.
- Let MD context switching run without worrying about spls or locks.
- Instead of having the idle loop implemented with special contexts
in MD code, implement one idle proc for each cpu. make the idle
loop MI with MD hooks.
- Change the proc lists from the old style vax queues to TAILQs.
- Change the sleep queue from vax queues to TAILQs. This makes
wakeup() go from O(n^2) to O(n)

there will be some MD fallout, but it will be fixed shortly.
There's also a few cleanups to be done after this.

deraadt@, kettenis@ ok

Use atomic operations to operate on netisr, instead of clearing it at splhigh.
This changes nothing on legacy architectures, but is a bit faster (and simpler)
on the interesting ones.

Send an IPI in signotify() if the process runs on a different processor,
similar to the fix which went into i386 and amd64 a few weeks ago.

In spl0(), really process soft interrupts at IPL_SOFT instead of whatever
level we were at.

Use atomic operations to change the pending software interrupt mask.

The softintr lock needs to be an __mp_lock, not a __cpu_simple_lock.

Replace gazillions of badvaddr() or badwordaddr() calls with badaddr() calls.
With a few prototype declarations shuffling, this finally allows
<machine/locore.h> to die.

Clean the internal m88k trap type codes; while there, simplify and
factorize the build of the VBR page betweem luna88k and mvme88k.

Tested by aoyama@ and I.

nmihand() is mvme88k only, move it out of m88k common area.

Factorize soft interrupt code between luna88k and mvme88k.

Oops, simplelock leftover.

Replace simplelocks with __cpu_simple_locks for cmmu and pmap locking,
for the MULTIPROCESSOR case.

Switch m88k ports to __HAVE_CPUINFO. Current cpu pointer is held in SR0
on all running processors.
Tested aoyama@ and I

Maintain an interrupt depth counter for CLKF_INTR to be correct.
Tested aoyama@ martin@ and I.

Invoke regdump() on abort if compiled without DDB support.

Merge <machine/cpu_number.h> into <machine/cpu.h>, preparing for intrusive
changes.

Remove commented out dead wrong FPU handling code in setregs().

Stop mapping the u area at fixed UADDR in addition to its actual va.
While there, attempt to clean and comment stack usage in the kernel.
No functional change.

From the m88k SMP tree; help&test martin@

Change the size parameter of cmmu_flush_tlb() from bytes to pages. This makes
things easier for the callers, and allows us to inline the "fewer than 4 pages"
situation for speed.

Use [XNF]IP_ADDR constants rather than their numeric values.

Introduce sparc-like CPU_ISFOO macros, to short-circuit evaluations when
compiling for one flavour only.

This makes code slightly smaller on all m88k-based platforms (my roughly
1KB), and saves more than meets the eye on luna88k, which is m88100-based.

More code shuffling, probably the final touch.

Kill SPINLOCK_SPIN_HOOK, use CPU_BUSY_CYCLE() instead.

ok visa@, kettenis@

Put the address of the per-cpu clock and interrupt registers into the cpu_info
struct, rather than fetching them from arrays every time they are needed.

ok aoyama@

Tweak the special handling of pid 1 in setregs()...and document that
while it's not 100% correct, it's okay

tested by aoyama@

Apply consistency to forever loops with continue and NOTREACHED

Same thought from kettenis@, ok krw@ phessler@ millert@

no md code wants lockmgr locks, so no md code needs to include sys/lock.h

with and ok miod@

Implement rw_cas() with atomic_cas_ulong() now, instead of a dedicated
routine.

Remove redundant regdump() output.

In regdump(), print DAE registers when they are valid, regardless of the
exception type.

Format string fixes for m88k; remove -Wno-format from the m88k kernels.

It's been a quarter century: we can assume volatile is present with that name.

ok dlg@ mpi@ deraadt@

Put the kernel VBR page at the beginning of the kernel text, as done on
mvme88k, to be able to unmap the page at address zero in the kernel, and have
the vbr page read-only after being initialized.

Fix setregs() to match prototype. Exposed by recent _types.h change, found the
hard way by aoyama@

Initialize a teeny few more globals to reasonable values to allow for printf()
to work as soon as cn_tab is initialized. Allows us to correctly printf or
panic early on unrecognized systems. This used to be the case but got lost a
long time ago.

Constify struct cmmu.

Make sure new processes get a clean FPU configuration upon startup, by
resetting the appropriate register in setregs(). Similar in spirit to what
m68k does (restoring a null frame), and necessary since m88k doesn't do lazy
fpu switching.

Switch m88k ports to ELF.

Make rusage totals, itimers, and profile settings per-process instead
of per-rthread. Handling of per-thread tick and runtime counters
inspired by how FreeBSD does it.

ok kettenis@

The exception vector page on m88k systems has always been the same page as
the one used by the firmware, which (at least on mvme88k) is at address zero.

This is unfortunate, since this means that NULL pointer dereferences in the
kernel are not caught, and writes cause havoc.

This behaviour was necessary to be able to use the PROM system call interface
during early bootstrap, without having to disassemble the VBR page and
update branches - which use pc-relative displacement - if we were to use
a different VBR address.

On mvme88k, change this and actually set up two VBR pages: one, over the PROM
page (except for the system call vectors), and another one in the kernel
image (which will be mapped read-only). We'll run with the PROM page until
the end of autoconf, and then switch to the kernel one, and unmap all the
PROM below it.

As a bonus, the final kernel page can use optimized data access fault
handlers, which do not have to check for faults caused by badaddr(), since
badaddr() is only used during device probe on buses which do not support
hotplug.

There are a few infrastructure collateral damage on aviion and luna88k, but
these ports do not change their behaviour yet.

Rename max_cpus to ncpusfound and compute it regardless of option
MULTIPROCESSOR.

Generic softinterrupt support for m88k based platforms, adapted from arm
with different locking mechanism. 88110 soft ipi are replaced with an
ipi callback which is checked upon return from exception (it can not be kept
as a softintr, as the generic softinterrupt code doesn't have per-cpu
pending softintr queues).

Introduce atomic_clear_int() as an MD atomic operation to perform atomic
exchange with zero; use it in the soft interrupt code to make it simpler
and faster.

Fix stupid logic bug in rw_cas_m88k(), makes NFS much happier on SMP kernels.

atomic_{set,clear}bits_int were not safe enough on 88110 systems, as they
can be interrupted by NMI; move the SMP version of these routines from
inlines to a separate file (kernel text shrinks 20KB...).

Since the implementation for 88110 becomes really hairy, the pre-main() code
is responsible for copying the appropriate code over for kernels configured
for both 88100 and 88110 cpus, to avoid having to choose the atomicity
strategy at runtime. Hairy, I said.

This gets GENERIC.MP run much further on 197DP. Not enough to reach multiuser
mode, but boots up to starting sshd and then panics.

typo

More 88110 SMP work. Contains, horribly entangled:
- dma_cachectl() split into a ``local cpu only'' and ``all cpus'', and an ipi
to broadcast ``local dma_cachectl'' is added.
- cpu_info fields are rearranged, to have the 88100-specific information
and the 88110-specific information overlap, and has many more 88110
ugly things.
- more ipi handling in the 197-specific area. Since it is not possible to
have the second processor receive any hardware interrupt (selection
is done on a level basis via ISEL, and we definitely do not want the
main cpu to lose interrupts), the best we can do is to inflict ourselves
a soft interrupt for late ipi processing. It gets used for softclock and
hardclock on the secondary processor, but since the soft interrupt
dispatcher doesn't have an exception frame, we have to remember parts
of it to build a fake clockframe from the soft ipi handler (ugly but
works).

This now lets GENERIC.MP run a few userland binaries before bugs trigger.

Proper cpu_unidle() function for MP kernels. ok art@ long ago

On 88100 cpus, make sure userland processes start with SNIP and SFIP valid
(i.e. with the valid bit set in them). Found the hard way by Anders Gavare
trying his latest gxemul, proves the hardware is more permitting than one
would expect it to be...

Remove the last debug bit from the PSR on 88110: do not force memory accesses
instructions to be serialized (this defeats the purpose of having a superscalar
processor, and accesses to volatile variables are done with explicit memory
barriers anyway).

This brings a HUGE speedup: openssl speed -elapsed shows AES is 90% faster,
blowfish is 75% faster, and sha1 is 50% faster. Not so bad!

However, doing this increases the pressure on the processor bus, so it is
necessary to increase the processor bus timeout on 40MHz boards again (to 256
usec). ``black cat'' 50MHz boards seem to be unaffected, so they remain at
64 usec.

Move the cmmu lock back from 8820x-specific code to global, and use it on
MVME197DP to serialize 88410 operations.

I honestly do not remember what 88110 errata convinced me to disable
out-of-order (superscalar) execution on these processors.

Since OoO brings a nice 50% to 250% speedup (as shown by ``openssl speed''),
it is definitely worth enabling.

xmem can not be used as a reliable atomic operation, they way we do them,
so do the naive operations with interrupts disabled, and an interlock held
if MULTIPROCESSOR.

Improve setregs() so that it performs the equivalent of the first two
instructions of the new binary, which allows them to be skipped unconditionaly.

When setting up the vectors page, do not put a nop as the first instruction
if the processor is a 88110, since only the 88100 suffers from the ``will
fetch one instruction too far'' bug.

Provide a specific rw_cas() function for MP kernels, simulating a really
atomic compare-and-swap operation.

Move the cmmu lock to 88200-specific code. 88110 MP code will use ipis
and will not require such a lock.

Fix setregs() on 88110, so that binaries do not skip their first instruction.
Makes binaries beyond init(8) running on MVME197, to some extent.

Replace many ``unsigned'' variables with ``unsigned int'', ``u_int'' or other
appropriate types. No functional change.

Check for curcpu()->ci_softintr being nonzero before invoking dosoftint(),
in the common case, this saves us two potentially expensive setipl() calls.

Resolve the struct trapframe tf_mode/tf_flags duality by getting rid of
the tf_mode name. This field only contains flags used internally by the
exception processing code.

Merge the ci_alive and ci_primary boolean values of struct cpu_info into
a single ci_flags bitfield.

Also, set_cpu_number() will no longer set CIF_PRIMARY on the primary processor,
it's up to the initialization code to do this.

On MULTIPROCESSOR kernels, don't forget to grab the kernel lock when
processing soft interrupts; and there was much rejoicing.

Keep a pending software interrupts mask per processor, instead of having it
global; and only schedule software interrupts on the currently running cpu.

Remove the now unused idle_u, and call the secondary processors startup
stack a startup stack.

When a secondary cpu gets its interrupt pin stuck, be sure to savectx
and put the process it was running back on the run queue (unless this was
the idle proc).

Do not flag a processor as ``alive'' until it really is ready to accept IPIs.

Remove sir_lock, superseded by the atomic bit operations.

Make context switching much more MI:
- Move the functionality of choosing a process from cpu_switch into
a much simpler function: cpu_switchto. Instead of having the locore
code walk the run queues, let the MI code choose the process we
want to run and only implement the context switching itself in MD
code.
- Let MD context switching run without worrying about spls or locks.
- Instead of having the idle loop implemented with special contexts
in MD code, implement one idle proc for each cpu. make the idle
loop MI with MD hooks.
- Change the proc lists from the old style vax queues to TAILQs.
- Change the sleep queue from vax queues to TAILQs. This makes
wakeup() go from O(n^2) to O(n)

there will be some MD fallout, but it will be fixed shortly.
There's also a few cleanups to be done after this.

deraadt@, kettenis@ ok

Use atomic operations to operate on netisr, instead of clearing it at splhigh.
This changes nothing on legacy architectures, but is a bit faster (and simpler)
on the interesting ones.

Send an IPI in signotify() if the process runs on a different processor,
similar to the fix which went into i386 and amd64 a few weeks ago.

In spl0(), really process soft interrupts at IPL_SOFT instead of whatever
level we were at.

Use atomic operations to change the pending software interrupt mask.

The softintr lock needs to be an __mp_lock, not a __cpu_simple_lock.

Replace gazillions of badvaddr() or badwordaddr() calls with badaddr() calls.
With a few prototype declarations shuffling, this finally allows
<machine/locore.h> to die.

Clean the internal m88k trap type codes; while there, simplify and
factorize the build of the VBR page betweem luna88k and mvme88k.

Tested by aoyama@ and I.

nmihand() is mvme88k only, move it out of m88k common area.

Factorize soft interrupt code between luna88k and mvme88k.

Oops, simplelock leftover.

Replace simplelocks with __cpu_simple_locks for cmmu and pmap locking,
for the MULTIPROCESSOR case.

Switch m88k ports to __HAVE_CPUINFO. Current cpu pointer is held in SR0
on all running processors.
Tested aoyama@ and I

Maintain an interrupt depth counter for CLKF_INTR to be correct.
Tested aoyama@ martin@ and I.

Invoke regdump() on abort if compiled without DDB support.

Merge <machine/cpu_number.h> into <machine/cpu.h>, preparing for intrusive
changes.

Remove commented out dead wrong FPU handling code in setregs().

Stop mapping the u area at fixed UADDR in addition to its actual va.
While there, attempt to clean and comment stack usage in the kernel.
No functional change.

From the m88k SMP tree; help&test martin@

Change the size parameter of cmmu_flush_tlb() from bytes to pages. This makes
things easier for the callers, and allows us to inline the "fewer than 4 pages"
situation for speed.

Use [XNF]IP_ADDR constants rather than their numeric values.

Introduce sparc-like CPU_ISFOO macros, to short-circuit evaluations when
compiling for one flavour only.

This makes code slightly smaller on all m88k-based platforms (my roughly
1KB), and saves more than meets the eye on luna88k, which is m88100-based.

More code shuffling, probably the final touch.