Ensure that the size of the stack is properly specified (2*USPACE) so it matches the allocation of stack pages done in pmap_bootstrap(). If this is not properly in sync, then the stack pointer may end up in the redzone right before the stack causing mmu faults. This issue has been observed on a T2000 (sun4v) system while probing the devices (pci/ebus/com) during bootstrap

sparc64: Optimization: Omit needless membar when triggering softint.

When we are triggering a softint, it can't already hold any mutexes.
So any path to mutex_exit(mtx) must go via mutex_enter(mtx), which is
always done with atomic r/m/w, and we need not issue any explicit
barrier between ci->ci_curlwp = softlwp and a potential load of
mtx->mtx_owner in mutex_exit.

PR kern/57240

XXX pullup-8
XXX pullup-9
XXX pullup-10

sparc64: Add missing barriers in cpu_switchto.

Details in comments.

PR kern/57240

XXX pullup-8
XXX pullup-9
XXX pullup-10

branches: 1.431.4;
fix various typos in comments.

fix various typos in comments.

fix various small typos, mainly in comments.

try to handle ultrasparc III* "fast ecc error" traps.

i had a sunblade 2500 reset with this trap recently, and the
manual says it should have been recoverable. in particular,
the US IIIv2 manual says in section 12.4.4.2 that the i-cache
won't see this data ever (so nothing needed there), and that
the d-cache needs to be disabled, and all the entries flushed,
and then re-enabled.

tested with a "trap 0x70" from kernel space, that trap.c
converted the "0x170" back to "0x70" (with an ADVANCE). this
code not commited.

Improve cpu_idle() by allowing a platform specific implementaion (same logic as the sparc implementation) - currently only used by sun4v

sun4v: handle spill trap properly when trap level is 1 and otherwin is non-zero - ensure that all windows are spilled to the pcb like the sun4u winfixsave code path

branches: 1.425.2;
sun4v: handle spill normal user traps properly when current trap level is 1 - registers are spilled to the PA of the stack to avoid further mmu-related traps, since sun4v only has 2 trap levels

sun4v: add comment in sun4v_tl1_ptbl_miss code path - no functional change

branches: 1.423.6;
oldlwp is always non-NULL in cpu_switchto so remove the test for NULL.

Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.

branches: 1.421.2; 1.421.4;
sun4v: added some missing comments to the code in pcbspill

make trap 0x70 a TRAP() not UTRAP(). we should be able to handle
fast ECC error as a normal trap, and i saw this once a few years
ago but it never repeated so i haven't been able to test this.

sun4v: Improve comment for next_stick_init() - no functional changes

sun4v: initialize %stickcmpr properly when establishing interrupt - fixes issue seen on newer sun4v/SPARC-T5 system where the interrupt never fires.

sun4v: introduce macros for handling spill traps from userlevevel @ traplevel 1 (from OpenBSD) - spill traps at level 1 from userspace now works better

sun4v: add comment documenting the contents of the %g2 register - no functional changes

Ensure that the 64-bit condition codes are used instead of the 32-bit condition codes. ok martin@

branches: 1.414.2;
next_stick() - fix typy in comment: tick->stick

sun4v: handle MMU-related traps properly when they occur at trap level 1. This fixes issues when user-level LWPs access the user-level stack

branches: 1.412.2;
sun4v: Avoid touching the %tick_cmpr register which is not present on never systems (Oracle SPARC Architecture 2011 and beyond). Based on code from OpenBSD locore.s revision 1.177. Tested on both sun4u (qemu and USIII) and sun4v (qemu and T5-2). ok martin@

branches: 1.411.2;
sun4v: The %ver register is not available on sun4v - use GET_MAXCWP introduced in rev. 1.361. Verified using qemu (sun4u and sun4v) and on real sun4v hw (t2000)

branches: 1.410.2;
sun4v: Add handling of trap 0x06c @ trap level 1 - based on code from OpenBSD, but slightly adapted to NetBSD. verified using qemu

sun4v: rft_user needs special handling (a manual fill) since the restore instruction may cause the number of trap levels to exceed the maximum for sun4v - inspired by code from the the openbsd rft_user code path - verified using qemu for both sun4u and sun4v

sun4v: Added handling of trap type 0x034 (address alignment error) + fixed typo mmfsa -> mmufsa. Verified for both sun4u and sun4v using qemu.

sun4v: Hook up syscall handling to the trap table. Verified using qemu for both sun4u and sun4v.

sun4v: implement missing handling of itsb traps 0x008 and 0x009. Based on code from OpenBSD. Tested using qemu.

branches: 1.405.2;
sun4v: slowtrap() now selects normal globals properly on both sun4u and sun4v platforms. Verified using qemu for both sun4u and sun4v systems

sun4v: TRAP_SETUP() updated to set mmu context id properly for both sun4v and sun4u platforms. Verified using qemu for both sun4u and sun4v systems

sun4v: make debugging kernel bringup a bit easier by using the slowtrap code path for currently unhandled trap level 1 trap entries 0x00-0x23 and 0x28-0x30

sun4v: make debugging kernel bringup a bit easier by using the slowtrap code path for currently unhandled trap level 1 trap entries 0x32-0x7f

Fix RAS for 32-bit kernels. trapframe is always 64-bit.

sun4v: rework sun4v_datatrap so it works properly when running on a T2-based system like T5120 - for some reason %tl is 0 when entering sun4v_datatrap causing a crash (sir instruction), so ignore the %tl value for now upon entry to sun4v_datatrap - with this fix both T1 and T2 based systems reach the same point - hangs in rft_user after the init process is started

sun4v: handle device interrupts properly - both legacy (sysino) and cookies

branches: 1.398.2;
sun4v: rtf_user now uses macros GET_MMU_CONTEXTID and SET_MMU_CONTEXTID. Tested on sun4u (QEMU) and sun4v (T2000). Parts from OpenBSD.

Decrement %tl in trap handler not to make it zero unconditionally
for the nested trap, and remove useless mov.

sun4v: Implement missng MMU protection trap handling - mostly from OpenBSD

Avoid assembler-specific defines by using the export keyword in genassym.cf instead

%g1 is not preserved across function call, so load it before
return_from_trap.

sun4v: add missing implementation of sun4v_datatrap_tl0 so data_access_fault() is called - mostly from OpenBSD

- ci_paddr is always 64-bit.
- intrlev is array of pointer.
- ih_ack is pointer.

sun4v: make device interrupts work. Introduce a new intrhand_alloc() function for allocation of interrupt handlers and adapt to this. Parts from OpenBSD. ok martin@

32-bit kernels use 32-bit stack.

sun4v: rft_kernel should pay attention to the number of register windows that can be restored. We cannot rely on fill traps to occur, since sun4v only has two trap levels available in privileged mode. Based on code from OpenBSD. ok martin@

sun4v: add breakpoint trap entry to the sun4v trap table

sun4v: extend dmmu_set_secondary_context() to handle both sun4u and sun4v. ok martin@

Basically revert r1.246 of locore.s and r1.73 of vm_machdep.c:

Remove special case handling for userland lwps from cpu_lwp_fork,
instead do it in lwp_trampoline when we first return to userland.

which was a stupid idea - since we did now set all fork child's %tstate
(and thus %pstate when back in userland) to the current kernel's userland
default. This meant we lost the address mask bit for 32bit processes and
all memory model details for 64bit ones.

Move it back to cpu_lwp_fork and fix the condition to only do it once when
forking init.

Most of the time return_from_trap already got called with tf->tf_tstate
loaded in register %g1 - make it so for two more callers and remove
the reload of that value.

sun4v: Ensure that the TTEs CP and CV bits are the same (both set) for the primary cpu and secondary cpus INTSTACK mappings

sun4v: Implement handling of cpu_mondo trap - from OpenBSD - tested using the Legion simulator

sun4v: revert some of the logic introduced in revision 1.378 - flushw is not necessary on sun4v (OpenBSD patches the flushw instruction to a nop instruction) - with these changes the kernel reaches cpu_boot_secondary_processors() when running in the Legion simulator (crashes due to missing handling of cpu_mondo_trap)

prom_printf converts "\n" to "\r\n" internally, so omit "\r".

PANICSTACK is a pointer itself, not a pointer to pointer.

Use panic stack temporarily for calling prom_printf in a trap
handler.

It seems that openfirmware cannot handle over 4GB pointers, and
stack pointers may point to over 4GB since VA layout was rearranged
by rev 1.33 of vmparam.h. Therefore, use panic stack to avoid
panicking in OF_write used by prom_printf.

sun4v: sparc_interupt - ensure that the trap level and globals level are the same when the flushw instruction is issued - othwise warnings occur when using the sun4v Legion simulator.

branches: 1.377.2;
sun4v: Create CLEANWIN0 and CLEANWIN1 macros for handling clean window traps on both sun4u and sun4u (based on the original code inlined in the sun4u trap table). Update trap tables accordingly for both sun4u and sun4v. Ok martin@

Fix typo in comment.

src is too big these days to tolerate superfluous apostrophes. It's
"its", people!

sun4v: fixed bug causing the real address to be incorrectly calculated as zero

sun4v: add code to enable startup of secondary cpus on both sun4u and sun4v systems - ok martin@

sun4v: All cpus must be setup with a TSB descriptor, so pmap_setup_tsb_sun4v() must take a pointer to the TSB descriptor

sun4v: Split the NORMAL_GLOBALS and ALTERNATE_GLOBALS macros to _SUN4U and _SUN4V variants. Rework the return_from_trap logic to use the new variants so it is possible to use a global register as scratch register.

Add missing delay slot in DEBUG kernel.

branches: 1.369.2;
Fix ALTERNATE_GLOBALS so it at least works on sun4u - using a scratch register is not a good idea...

sun4v: Introduce a new macro ALTERNATE_GLOBALS (matching NORMAL_GLOBALS). Two new helper macros ENABLE_INTERRUPTS and DISABLE_INTERRUPTS are also created. With these changes the GENERIC.UP kernel will reach rootmount on sun4v systems - ok martin@

Improved the NORMAL_GLOBALS macro in the sun4u case, so only the relevant bit (AG) in %pstate is affected - ok martin@

sun4v: add missing spill/fill handling for trap level 1 + cosmetic changes to comments for trap level 0 entries

sun4v: Introduce a NORMAL_GLOBALS macro - first used in sparc_interrupt - other relevant uses will be handled as the sun4v work is progressesing - ok martin@

sun4v: Introduce SET_MMU_CONTEXTID macro for setting the mmu context id on both sun4u and sun4v

sun4v: add interrupt handling to sun4v trap table

Bugfix: add missing cmp instruction to GET_MAXCWP macro

sun4v: The maximum number of register windows is a constant and thus not available via the %ver register (sun4u only). Introduce a macro GET_MAXCWP that handles this, so cpu_switchto() works properly on sun4v. Other relevant reference to %ver can be adapted to use GET_MAXCWP as the sun4v port progresses. OK martin@

sun4v: added missing dtsb miss handling for trap level 1 - with this update the kernel now reaches configure() on sun4v systems

Remove debugging code introduced by accident in locore.s revision 1.357

sun4v: the trap base address should be assigned to %l1 so %tba is properly setup later in the code - now the kernel gets as far as printing the copyright disclaimer, kernel version and total and available memory before crashing

sun4v: Update comment: modified -> access

branches: 1.356.2;
sun4v: Rename TLB_ defines to SUN4U_TLB_ so entries created using TSB_DATA() are properly setup for sun4u and sun4v. Relocate the cputyp variable from autoconf.c to locore.s and make it const in param.h so optimized code can be generated. Parts from OpenBSD. Optimization suggested by nakayama@. OK martin@, mrg@, nakayama@

defined(LKM) -> defined(MODULAR)

sun4v: Add handling of spill/fill and dtsb miss traps (with a XXX to be handled later)

sun4v: trap table setup - currently populated with dummy entries which will be properly implemented later - parts from OpenBSD - OK martin@

Make buildable w/o options SUN4V.

a) make the hunt through the cpus list more generic b) sun4v: ensure that the interrupt stack is mapped permanently and the TSB is setup properly - parts from OpenBSD. OK martin@

Increase an interrupt depth only in the case of hardware interrupts,
and remove the ci_idepth trick in softint_fastintr.

Fixes the following diagnostic panic reported in port-sparc64.

panic: kernel diagnostic assertion "!cpu_intr_p()" failed: file
"../../../../kern/subr_xcall.c", line 351

branches: 1.349.2;
Disable interrupts after calling data_access_error as with
data_access_fault.

Use macros in machine/asm.h instead of locally defined, and
introduce CASPTRA for casxa/casa.

Fix sparc64_ipi_ccall by adding proper trap setup.
Fixes xcall(9) problems, as exposed by pserialize(9). Noticed by
J. Hannken-Illjes, cause pointed out by Takeshi Nakayama.

Sligthly clean up the (by default unused) #ifdef TRAPS_USE_IG

Revise comment.

Use macro instead of magic number.

Make setstick() and getstick() to inline functions.

use system tick timer instead of %tick on UltraSPARC-III-ish CPUs
review & fixes by Takeshi Nakayama

branches: 1.341.2;
port the corrected ECC error handling code from freebsd. i noticed my U10
took one of these and then hang. it shouldn't hang, there's a 'sir' here that
doesn't seem to reset properly. oh well.

tested by simulated a trap via 'ta 0x10' and considering that the same, but
it hasn't been tested against a real ECC error yet.

Fix the root cause of the hack "disable optimizations for uvm_bio.c
on 32 bit kernels".

gcc converts a division in the calculation of UBC_UMAP_ADDR macro
to multiplication (smul or combination of add/sll), and the
register of its result contains a garbage in upper 32 bits (the
upper 32 bits of smul/add/sll's result isn't zero cleared).

Then it passes to pseg_get{,_real} through pmap_extract without the
zero clear of upper 32 bits in the optimization case. So the
result of pseg_get and pmap_extact sometimes gets screwed up.

Remove COMPAT_SA / KERN_SA. Welcome to 6.99.3!
Approved by core@.

branches: 1.338.2; 1.338.6; 1.338.8;
ih_pil is an interrupt level, not an interrupt mask.

ivec is assigned to hardware, so count its event counter up in
interrupt_vector, not in softint, and do some optimization.

add per ivec event counters

rename sparc64 BLOCK_SIZE and BLOCK_ALIGN to have SPARC64_ prefixes.
for the assembler files, define the old names to the new names
since using the new names cause ugliness due to longer identifer
names, and reduces churn.

fixes build issues in dtv and vaguely makes <machine/psl.h> slightly
less name-space invasive.

Add fast softint(9) support for sparc64.

Reviewed on port-sparc64.

branches: 1.333.2;
first steps towards fujitsu SPARC64 support:

- SPARC64 systems have interrupts with upto 14 bits set, but we only look
at the first 11 bits. trim non IPI interrupts to 11 bits

branches: 1.332.2;
Move counting of faults, traps, intrs, soft[intr]s, syscalls, and nswtch
from uvmexp to per-cpu cpu_data and move them to 64bits. Remove unneeded
includes of <uvm/uvm_extern.h> and/or <uvm/uvm.h>.

Clear the upper word explicitly before combining two 32-bit values to 64-bit
on 32-bit kernels.

Split copy related functions out of locore.s into copy.S, move functions
only needed on multiprocessors into mp_subr.S.

Simplify primary CPU startup slightly by using the fact that our cpuinfo's
curpcb pointer points to the uarea/pcb of lwp0 and by remembering the pointer
to this cpuinfo, thus avoiding a few CPUINFO_VA references.

fix copy{in,out}{,str}() to return the error returned by uvm_fault().
fixes PR 41813.

branches: 1.327.2;
XXX: workaround we'd like to remove when pmap / uvm locking is cleaned up:

- rename pseg_get() and pseg_set() to pseg_get_real() and pseg_set_real().
- if USE_LOCKSAFE_PSEG_GETSET is defined, which it current is by default,
define pseg_[gs]et() in terms of functions that take a new pseg_lock
mutex at IPL_VM while calling into the real functions.

this seems to avoid the pseg_set() crashes we've seen:
1 - spare needed, when pseg_get() just worked for this pmap
2 - the 2rd ldxa via ASI_PHYS_CACHED in pseg_set() loads garbage
into %o4, and causes the 3rd ldxa to fault

blast_dcache() SMP friendly:

- blast_dcache() becomes sp_blast_dcache(dcache_size, dcache_line_size)
- new smp_blast_dcache(sparc64_cpuset_t) that blasts the D$ on this cpuset
- sparc64_ipi_blast_dcache() to support the above
- in pmap_remove_all(), when freeing mmu contexts for this pmap, mark
the set of cpus to blast the d$ on as well and convert the
blast_dcache() call into smp_blast_dcache() on the cpus who ran this
pmap, or, sp_blast_dcache(dcache_size, dcache_line_size)
- convert the remaining blast_dcache() in machdep.c to sp_blast_dcache()
- in pmap_destroy()/pmap_remove_all() take the pmap_lock() always since
we assert it is held always.

with these changes, NFS builds on the U60 seem to be stable now, and
the USIII machines also can often complete a single build.sh run now,
diskful or diskless.


reviewed by mlelstv and partially by martin, tested by martin and myself,
with some ideas from chuq as well.

need _ALIGN in locore.h as well, for DEBUG kernels.

gas has supported SPARC ".empty" for a long while; get rid of an
ancient workaround.

move the memcpy/memset implementations out into their own file, with the
block copy versions as well. move some of the definitions in locore.s
into a new locore.h.

locore.s is almost 2000 lines shorter with this change.

- in _bus_dmamap_unload(), pmap_page_protect() and pmap_clear_reference()
switch the dcache_flush_page() into a dcache_flush_page_all()
- in both pmap_kremove()/pmap_remove(), remove the blast_dcache() call
and replace it with dcache_flush_page_all()
- in pmap_get_page() [used to allocate PTP's], always call pmap_zero_page(pa)
- flush the dcache of the dst page in pmap_{copy,zero}_page() by redirecting
throught a C function that calls the (renamed) asm. the old asm code had a
comment about needing to do this...
- add a couple of membar #Sync's that the USIII manual recommends


based on discussions with chuq@, skrll@ and martin@.


these help my SB2000 / SB2500 with both disk / nfs builds and other tasks,
sometimes lasting for several hours before failing or asserting.

when handing back page table pages to UVM, flush them from the dcache
on all CPUs.

idea from skrll@, and makes my disk-based SB2000 stable.

tested on U60, SB2000 and SB2500.

- re-introduce sp_tlb_flush_pte() and sp_tlb_flush_all() that call
the right USI/USIII version.
- define tlb_flush_pte() in terms of sp_tlb_flush_pte() for the
!MULTIPROCESSOR case

this centralises some CPU_IS_USIII_UP() tests.

move {dump,print}_[id]tlb() into a separate file.

- store the maximum [id]cache size / line size for each cpu

- remove patch_kernel() since it isn't necessary and has been patching
the wrong thing in general.

- implement USIII versions of blast_icache(), dcache_flush_page() and
cache_flush_phys(). use the newly recorded cache size/line sizes.

- in winfixsave, flush the D$ with the right size index and ops. this
kills one of the wrong tag->inval patches.

- for blast_dcache(), use the newly recorded dcache_size/line_size.

- for blast_icache(), use the newly recorded cache size/line sizes.
for the USIII verison, disable the I$ while writing to ASI_ICACHE_TAG.


these changes removed several hard coded cache sizes values, some very
wrong kernel patching, and seem to make the current failure modes for
USIII less common, but not gone.

- don't flush 32-bit entries for 64-bit code, it does not seem to be
necessary. #define TLB_FLUSH_LOWVA to turn it back on.
- remove the #if 0'd code in sparc64_ipi_flush_pte_usiii(), i have
verified that it is not required.
- add a missing membar #Sync to sp_tlb_flush_pte_usiii().

Fix linker error of GENERIC32 kernel.

use TLB_SIZE_SPITFIRE in a couple of places, and note that the
{print,dump}_[id]tlb() versions are currently spitfire specific.

introduce a new patch_kernel() and call it from the end of bootstrap().
use it to patch up TLB flush instructions that don't match for USI/II
and later. we can also hopefully use this to make GENERIC.MP minimally
expensive on single CPU machines as well.

this is the last piece needed to get GENERIC/GENERIC.MP to boot on the
USIII* machines.

idea and some initial code from openbsd.

remove KTR support. it's never been a working part of netbsd src tree.

Fix typo in comment.

branches: 1.311.2;
- properly map the schizo interrupt registers, so we can use the normal
virtual mappings to access them.
- with the above fxied, kill SCHIZO_BUS_SPACE_BROKEN, and introduce
an options CHEETAH for now, that disables the SPITFIRE code in locore.
- merge schizo_init() into schizo_attach() and clean up the softc a bit.


with this we are only one more step away from GENERIC booting on USIII
machines.

- split sp_tlb_flush_pte() and switchtoctx() into sp_tlb_flush_pte_us()/
sp_tlb_flush_pte_usiii() and switchtoctx_us()/switchtoctx_usiii() and
implement the latter while i'm here. it works ... sometimes i think,
but also sometimes panics/hangs.
- fix a comment in sparc64_ipi_flush_pte_usiii()

for now, reduce a few #ifdef SPITFIRE cases by introducing a new
ASI_DCACHE_TAG_OR_INV that is either ASI_DCACHE_{TAG,_INVALIDATE}.

increase the common code in cache_flush_phys().

- sp_tlb_flush_all() becomes sp_tlb_flush_all_us() and sp_tlb_flush_all_usiii()
- sparc64_ipi_flush_pte() becomes sparc64_ipi_flush_pte_us() and
sparc64_ipi_flush_pte_usiii()
- add some commented out code to disable interrupts and raise the traplevel
in sparc64_ipi_flush_pte_usiii()
- cache_flush_phys() was missing a little of code in the cheetah case

for USIII, use the new "tlb demap all" operation in sp_tlb_flush_all()

remove unused icache_flush_page() implementation.

remove the final uses of DATA_START.

now we can remove the 16KB space reserved at the start of the data segment,
and i now understand why i wrote this now removed comment 3.5 years ago:
"this shouldn't be needed... but kernel usually hangs without it"

remove most of the code that writes debugging info DATA_START + offset.

remove CHKPT() macro and uses. it's likely wrong in SMP land and probably should
be replaced with KTR-style uses if necessary.

fix a comment.

Rename L_ADDR to L_PCB and amend some comments accordingly.

- adjust the PA mask to the 43 bits supported by USIII.
- rework the spitfire/cheetah cache_flush_phys() to avoid several #ifdefs
- add a membar #Sync after sparc64_ipi_drop_fpstate (probably unnecessary)

sp_tlb_flush_pte for cheetah:
- restore interrupts properly in case of %tl > 0.

stablize UP USIII support by blocking interrupts around sp_tlb_flush_pte()
i was seeing stack corruption while taking an interrupt in this function.

get USIII SMP mostly working by implementing the cheetah version of
sparc64_ipi_flush_pte().


SMP support is still not entirely stable. i can reproducably get a:

panic: fpusave_lwp ipi didn't

while running build.sh, when an awk process is exiting. other simple
heavy work loads do not crash for me right now.

delete some no longer used code:
- smp_tlb_flush_ctx()
- smp_tlb_flush_all()
- sparc64_ipi_flush_ctx()
- sparc64_ipi_flush_all()
- sp_tlb_flush_ctx()

- calculate end PA correctly.
- wrap more spitfire specific (flushing I$) code.
- fix inverted #if condition for flushing I$ (and restore ASI_ICACHE_TAG).

implement enough code to get SB2000/SB2500 booting to multiuser. does
not last long if you use it, but an idle machine has stayed up 10 hours.

- for the !SPITFIRE case, use ASI_DCACHE_INVALIDATE instead of
ASI_DCACHE_TAG when clearing tlb entries. from openbsd.

- convert some comment "! Not yet" into "WRITEME", so that attempting
to build a USIII MULTIPROCESSOR kernel fails to build until the code
is written.

- add some more WRITEME's for the above.

- implement a !SPITFIRE version of sp_tlb_flush_all(). since there are
no spare regs, for now just always increase %tl around this call. (we
could maybe store a token in %o3 along with the saved %pstate info.)

Kill proc0paddr. Use lwp0.l_addr instead.

s/L_ADDR/L_PCB/, requested by rmind.

Wrap unused code with #if 0 which is called from #if 0'ed code.

cpu_setfunc() can not use lwp_trampoline, as that has additional lwp startup
semantics. Use a simpler setfunc_trampoline instead.

Remove random() from locore.s since it is in libkern.

Remove clear %tick in locore.s since it is cleared at cpu_initclocks() in
clock.c.

restore some old "intrdebug" code, but leave it under NOT_DEBUG. most
of it works, but one that prints func/arg for each handler is not very
stable...

for now, while schizo bus space pointer accesses aren't working, add
a SCHIZO_BUS_SPACE_BROKEN option so you can build a kernel with schizo.

with this, and the prior schizo interrupt fix, you can now fail to run
/sbin/init!

Get rid of get_maxctx() - it doesn't work on newer cpus and we will have
to find a better way for (some) sun4v machines - all sun4u use 13 bit
contexts anyway.

branches: 1.286.2; 1.286.6;
Wrap more code specific to USE_BLOCK_STORE_LOAD.

branches: 1.285.2;
- in flushing cache, use 32 as increment/decrement value to match the
cache line size.
- remove useless restore of %asi.
- remove nop in delay slot if we can.

Use proper CCR.

sparc64_ipi_save_fpstate:
- use primary MMU context for consistency with other trap/interrupt handlers.

sparc64_ipi_save_fpstate, savefpstate:
- avoid storing fp registers as we can.

sparc64_ipi_save_fpstate, savefpstate, loadfpstate:
- remove unaligned case since buffers allocated with pool_cache are ensured
64-byte aligned.

Ok by martin@.

branches: 1.282.2;
UltraSPARCs don't have floating point queue, and reading %fq regsiter
will cause illegal instruction trap. So remove the code handling them.

Wrap IPI calls with #ifdef MULTIPROCESSOR.

Close unterminated comment.
Fix unexpected behavior in case of loadfpstate from unaligned buffer.

branches: 1.279.2;
openfirmware_exit:
- disable interrupts with %pstate.
- prepare 64-bit stack and enable 64-bit addresses for 32-bit SMP kernels.

branches: 1.278.2; 1.278.4; 1.278.6;
Disable TRAPSTATS which was accidentally enabled when yamt-idlelwp branch
was merged.

Ok by martin@.

Remove sparc64_ipi_sync_tick.

Since we can use counter-timer as timecounter instead of %tick on SMP kernel,
it is not necessary to sync all CPUs %tick registers.

Add workaround for BB_ERRATA_1 on writing to TICK_CMPR register in
next_tick(). If writing to TICK_CMPR fails, we lose hardclock interrupt
on secondary CPUs.

About BB_ERRATA_1 from comment in OpenSolaris:

/*
* Writes to the TICK_COMPARE register sometimes fail on blackbird modules.
* The failure occurs only when the following instruction decodes to wr or
* wrpr. The workaround is to immediately follow writes to TICK_COMPARE
* with a read, thus stalling the pipe and keeping following instructions
* from causing data corruption. Aligning to a quadword will ensure these
* two instructions are not split due to i$ misses.
*/

Don't read fplwp content in delay slot for considering the case fplwp was
modifiled.

Reported by Chris Ross on port-sparc64.

- remove disabling interrupt in IPI.
- unused VA bits in MMU demap is 13 bits, not 12 bits.

Fix a race when increment an interrupt level event counter.

Fix a race condition in IPI handlers of save/drop fpstate, from OpenBSD.
Reported and tested by martin@.

Make schedintr interrupt handler per-CPU.
While there rename tickintr interrupt handler and share initialization
code with schedintr.

Improve IPI handling:
- make IPI takes two arguments.
- add IPI event counters per-CPU.
- implement IPI functions which were missing or broken.
- insert DELAY while halting primary CPU in IPI handler.

- make interrupt pending list per-CPU.
- make tickintr() MP-safe.
- remove unused port-sparc derived interrupt code.

Ok by martin@.

Don't get IPI handler argument (from bogus address), and ack IRQ twice.

Make TSBs and MMU contexts per-cpu.

branches: 1.266.2; 1.266.6;
Fix for 32-bit kernels.

Store a sensbile PCB_PC value in snapshot - the return point.

This helps gdb somewhat.

OK'd by martin.

Fix register name typo/copy&pasto - %g6 is not %o6/%sp.
From OpenBSD.

Load the next cpu_info pointer with LDPTR instead of ld.

branches: 1.262.2;
Provide cpu_intr_p(), at least for non-MULTIPROCESSOR kernels.
Based on suggestions by Andrew Doran.

branches: 1.261.2; 1.261.4; 1.261.6;
Remove an unused .globl declaration

Merge the ppcoea-renovation branch to HEAD.

This branch was a major cleanup and rototill of many of the various OEA
cpu based PPC ports that focused on sharing as much code as possible
between the various ports to eliminate near-identical copies of files in
every tree. Additionally there is a new PIC system that unifies the
interface to interrupt code for all different OEA ppc arches. The work
for this branch was done by a variety of people, too long to list here.

TODO:
bebox still needs work to complete the transition to -renovation.
ofppc still needs a bunch of work, which I will be looking at.
ev64260 still needs to be renovated
amigappc was not attempted.

NOTES:
pmppc was removed as an arch, and moved to a evbppc target.

branches: 1.259.4;
Make cpufrequency and friends per cpu values.
Prepare a hz tick interrupt on secondary CPUs via %tick, but do not
enable it yet, as it breaks ddb.

Do not ACK IPIs before we finished loading all arguments.

Remove the (now unused) second 64k page mapped per CPU.
From matthew green, with small changes by me. All bugs are mine.

branches: 1.256.2;
Remove INITSTACK completely - at the time we used to switch to it, we
already have access to all of lwp0 and it's uarea - so we can switch
to the correct lwp0 stack easily before calling main.

When exiting to OF, use the interrupt stack instead of INITSTACK

Rework cpu_mp_startup slightly:
- temporarily use the cpu's interrupt stack untill we are ready
to access non-OF-based mappings, then directly switch to the cpu's
idlelwp's uarea stack
- disable interrupts untill we are ready to call C code

remove two restore without matching save

branches: 1.252.4; 1.252.8;
A few nits for 32bit SMP kernels, noticed by mrg.
While there, simplify the fpstate IPIs a bit.

sparc64_ipi_save_fpstate is not a leaf function - create a full grown
stack frame.

clean up cpu_switchto() a little - no need to flushw or membar
excessively. optimise an rdpr. tested by tnn.

Fix a stupid bug in the save_fpstate ipi, which caused overwriting of
struct lwp of the lwp owning the fpu. Fix clearing the fplwp - %o0 will
not survive the call to savefpstate - duh!

Simplify tf_tstate setting when leaving for userland initially - I seem
to have confused the if with the else case of the previous C code.
Funny that it worked at all.

cpu_switchto() has to return it's first arg, always. This differs from
my reading of the documentation, but helps in the real world and seems
to be what other ports do.

Remove special case handling for userland lwps from cpu_lwp_fork, instead
do it in lwp_trampoline when we first return to userland.

cpu_idle is a leaf function, so we better "return from leaf".
Use return value from cpu_switchto for previous lwp in lwp_trampoline.
Also shamelessly steal a comment from uwe written for sparc that explains
all this.
Thanks to tnn, mrg, and uwe for helping to debug this.

merge yamt-idlelwp branch. asked by core@. some ports still needs work.

from doc/BRANCHES:

idle lwp, and some changes depending on it.

1. separate context switching and thread scheduling.
(cf. gmcgarry_ctxsw)
2. implement idle lwp.
3. clean up related MD/MI interfaces.
4. make scheduler(s) modular.

branches: 1.243.4; 1.243.6; 1.243.12;
uvm.page_idle_zero now is a bool, not a 32bit value any more.

shave 9 insns off the tsb setup in cpu_initialize & cpu_mp_startup.

Get rid of unused (thus always zero) "curlwp" variable hidden in assembler
code (C code always uses a cpuinfo based macro).

Make use of another call delay slot, pointed out by uwe.

Make use of a call delay slot and avoid a bogus EMPTY.
Spotted by Juergen Hannken-Illjes, with input from Andrew Doran and
Valeriy E. Ushakov.

Whitespace cosmetics

Backout previous: EMPTY was misused

Make EMPTY expand to nop with gas, as it is used to mark empty branch
delay slots.

branches: 1.235.2;
Fix obvious copy&pasto in SCHED_DEBUG register usage.

Merge newlock2 to head.

Spell "relevant" correctly. From Zafer Aydogan.

branches: 1.232.2; 1.232.4;
- note that __idle_u should go away but doing so breaks things
- fix several comments to reality
- clean up the #ifdef NO_VCACHE code slightly
- use call instead of set/jmpl
- remove a couple of unnecessary stores to curlwp

Clean up some comments. Cosmetic change to use correct structure
offset names.

Optimize out two instructions from sparc64_ipi_save_fpstate and
sparc64_ipi_drop_fpstate.

allocate 128KB (as opposed to 64KB) per-cpu, and place the interrupt
stack, cpu_info structure, panicstack, idle uarea and normal kernel
stack in here. the idle uarea and kernel stack are in the 2nd 64KB
page. the interrupt stack and cpu_info structure are as the same
place in the 1st 64KB page, and the panicstack grows down from the
top of this page. update comments in param.h to suit (XXX: a lot
of these constants should move to a header file that will not pollute
the name space so much.) map these pages identically between
cpu_initialise() and cpu_mp_startup(), and deal with the two 64KB
pages in pmap_extract(). keep an unmapped 64KB page between them.

don't initialise the unused "ci_eintstack" member of the cpu_info
structure

use constants to initialise many addresses in the cpu_info structure,
consolidating these assignments from cpu0 and cpu>0

delete u0/estack0/panicstack from locore.s

clean up some wrong comments in locore.s

fix a 32bit-only bug in cpu_mp_startup().

delete 'u0' entirely.


this fixes recent random lossage seen on port-sparc64: processes
stuck in RUN state, data faults, and hangs.

use call instead of set/jmpl. add an entry point for idle that expects
sched_lock to be unlocked, and use this for non-boot cpu's. complete a
comment describing how registers in idle_switch are expected to be set.

use the same SAVE_LOCALS_INS definition for 32 bit and 64 bit kernels.
refactor TRAP_SETUP() a little between these two versions to be more alike.

use the correct size/type for MP variables (pointers and longs).

a 32 bit MULTIPROCESSOR kernel now builds (and crashes quickly.)

in the 64bit version of SAVE_LOCAL_INS, s/BIAS/STKB/ to allow future
sharing of the 32 and 64 bit versions of the {TRAP,INTR}_SETUP macros.

- remove some way old FP setup/teardown code
- put the block memcpy/memset inside #ifdef USE_BLOCK_STORE_LOAD which
is (now) defined.

no functional change, same .o is generated.

re-order the changes in rev 1.217 to give registers a little bit of
time to settle between usage, like it was before hand.

- remove a wrong comment.
- add a new entry point for idle called idle_switch that is called by
cpu_switch() in the MP case when sched_whichqs is zero, properly
setting up the idle stack. from petrov@ via martin@.

rework interrupt_vector() slightly:
- for the IPI case, avoid loading the address of intrlev into %g3
- remove some NOT_DEBUG code
- load the interrupt argument #2 in the delay slot

this removes 2 nops during interrupt processing for the normal
case, and an additional 2 ALU instructions for IPIs

- sparc64_ipi_halt(): we shouldn't return from sparc64_ipi_halt_thiscpu()
so replace a branch with a 'sir'
- in #if 0 code, in sparc64_ipi_flush_pte() and sparc64_ipi_flush_ctx()
save/restore the out registers that the sp_tlb_flush_xxx() versions
around calling them
- in sp_tlb_flush_pte() and sp_tlb_flush_ctx(), replace global register
usage with out registers so that we don't clobber the (alternate)
global registers when these are called from IPI context

- remove an extra 'sir' that followed one in a macro
- add some named labels to some spill/fill handlers
- remove 2 unused strings

branches: 1.219.2;
Lazy FPU handling for the MULTIPROCESSOR case

From OpenBSD:
- reload %fprs for checking it again.
- use symbolic names for checking in %fprs.

revision 1.51
date: 2005/07/18 14:50:11; author: deraadt; state: Exp; lines: +2 -1
cache a copy of the fprs so we know what fp restore we should do after

revision 1.50
date: 2005/07/14 01:46:13; author: deraadt; state: Exp; lines: +4 -4
use symbolic names for checking in %fprs, as above

Fix a couple of %y register save/restore errors, inspired from OpenBSD.

revision 1.44
date: 2004/04/23 04:18:17; author: marc; state: Exp; lines: +5 -4

fix a couple of %y register save/restore errors
1) don't save %y in a register that is later used before restoring %y
2) always restore %y after calling a signal handler
tested by drahn@, OK pval

Do not bother to save fpu state when we are about to get rid of it.

SMP cleanup. provide support for multiple CPUs in DDB. (SMP itself
is still not working.)

cpu.h:
- add a pointer for DDB regs in SMP environment to struct cpu_info
- remove the #defines for mp_pause_cpus() and mp_resume_cpus()
cpuset.h:
- remove CPUSET_ALL() and rename CPUSET_ALL_BUT() to CPUSET_EXCEPT()
from petrov.
db_machdep.h:
- rename the members of db_regs_t to be the same as sparc
- change "db_regs_t ddb_regs" to "db_regs_t *ddb_regp" and change
all references to suit
- redo DDB_REGS to no longer be a pointer to a fixed data structure
but to one allocated per-cpu when ddb is entered
- move a bunch of prototypes in here
intr.h:
- remove SPARC64_IPI_* macros, no longer used
db_interface.c:
- change "db_regs_t ddb_regs" to "db_regs_t *ddb_regp" and change
all references to suit
- make "nil" a 64 bit entity
- change the ddb register access methods to work in multiprocessor
environment, it is now very much like sparc does it
- in kdb_trap() avoid accessing ddb_regp when it is NULL
- update several messages to include the cpu number
- unpause other cpus much later when resuming from ddb
- rename db_lock() to db_lock_cmd(), as the sparc-like code has
db_lock as a simple lock
- remove "mach cpus" command, and replace it with "mach cpu" (which
does the same) and also implement "mach cpu N" to switch to
another cpus saved trapframe
db_trace.c:
- update for the ddb_regs -> ddb_regp change
genassym.cf:
- add TF_KSTACK as offsetof(struct trapframe64, tf_kstack)
ipifuncs.c:
- overhaul extensively
- remove all normal interrupt handlers as IPI's, we now handle
them all specially in locore.s:interrupt_vector
- add a simplelock around all ipi functions - it's not safe for
multiple cpus to be sending IPI's to each other right now
- rename sparc64_ipi_pause() to sparc64_ipi_pause_thiscpu() and,
if DDB is configured, enable it to save the passed-in trapframe
to a db_regs_t for this cpu's saved DDB registers.
- remove the "ipimask" system (SPARC64_IPI_* macros) and instead
pass functions directly
- in sparc64_send_ipi() always set the interrupt arguments to 0,
the address and argument of the to be called function. (the
argument right now is the address of ipi_tlb_args variable, and
part of the reason why only one CPU can send IPI's at a time.)
don't wait forever for an IPI to complete. some of this is
from petrov.
- rename sparc64_ipi_{halt,pause,resume}_cpus() to
mp_{halt,pause,resume}_cpus()
- new function mp_cpu_is_paused() used to avoid access missing
saved DDB registers
- actually broadcast the flush in smp_tlb_flush_pte(),
smp_tlb_flush_ctx() and smp_tlb_flush_all(). the other end may
not do anything yet in the pte/ctx cases yet...
kgdb_machdep.c:
- rework for changed member names in db_regs_t.
locore.s:
- shave an instruction from syscall_setup() (set + ld -> sethi + ld)
- remove some old dead debug code
- add new sparc64_ipi_halt IPI entry point, it just calls the C
vector to shutdown.
- add new sparc64_ipi_pause IPI entry point, which just traps into
the debugger using the normal breakpoint trap. these cpus usually
lose the race in db_interface.c:db_suspend_others() and end up
calling the C vector sparc64_ipi_pause_thiscpu().
- add #if 0'ed code to sparc64_ipi_flush_{pte,ctx}() IPI entry
points to call the sp_ version of these functions.
- in rft_kernel (return from trap, kernel), check to see if the
%tpc is at the sparc64_ipi_pause_trap_point and if so, call
"done" not "retry"
- rework cpu_switch slightly: save the passed-in lwp instead of
using the one in curlwp
- in cpu_loadproc(), save the new lwp not the old lwp, to curlwp
- in cpu_initialize(), set %tl to zero as well. from petrov.
- in cpu_exit(), fix a load register confusion. from petrov.
- change some "set" in delay branch to "mov".
machdep.c:
- deal with function renames
pmap.c:
- remove a spurious space
trap.c:
- remove unused "trapstats" variable
- add cpu number to a couple of messages

significant clean up effort, largely from petrov@ via martin@.

- fix a bunch of comments for proc->lwp
- remove old TRAPTRACE, NOT_DEBUG, etc. code that has bit-rotted
- put some older DEBUG code under NOT_DEBUG
- add a bunch of new debugging/tracing code ("KTR") that depends on
other changes not yet present.

branches: 1.213.6; 1.213.8;
integrate kauth.

Fix some comments.

Load CPCB in cpu_mp_startup.

branches: 1.210.2; 1.210.4; 1.210.6;
ANSIfication: u_intN_t -> uintN_t, use ANSI function declarations/definitions
instead of K&R ones.

branches: 1.209.2; 1.209.4;
Alter sparc64 bootstrap, catch up to ofwboot v1.9:

- Accept bootinfo structure passed down from ofwboot v1.9
- Drop kernel re-mapping code
- Use permanent 4MB mappings provided by the loader instead
- Change kernel entry address to point directly at the code instead of pointing
at the trap table's first slot. This allows the bootloader to detect
those kernels which are aware of the new boot scheme
- Due to the changes in kernel mapping code, alter secondary CPU bootstrap
code to use trampoline just like FreeBSD does (some FreeBSD code is used
here as well)

Fix a typo: %g0 should be used instead of %o0 when initializing FPU in dostart:
and cpu_mp_startup:, %o0 is not even initialized at this point.

branches: 1.207.2;
merge ktrace-lwp.

Re-factor syscall, and make it use syscall_{plain,fancy}.

branches: 1.205.2;
remove vestigal {,e}intr{names,cnt} symbols now that vmstat doesn't need them.

branches: 1.204.4; 1.204.6; 1.204.8;
in sparc_interrupt(), wait to enable interrupts until just before calling
the handler and disable them again right after the handler returns.
we don't want another hardware interrupt coming in while we're mucking
with the list pointers. also, clear the intrhand's ih_pending pointer
before calling the handler rather than after, so that if another interrupt
comes in while the handler is running, we can put the intrhand back on
the list so it will be called again.

in send_softint(), disable interrupts via %pstate instead of %pil, so that
hardware interrupts at higher levels are blocked while we manipulate the
intrpending list. also, if the intrhand already has ih_pending set,
don't bother poking the SET_SOFTINT register again, since being on the list
implies that there is already a softint pending in the hardware.

these appear to fix PR 21750 (the "sleep forever" problem).

while I'm here, remove more vestiges of the !INTRLIST softint stuff
(code and comments). and in sparc64_ipi_flush_all(), interrupts are
already disabled so we don't need to fiddle with %pstate.

Remove the no long usefull variants selected by undefining INTRLIST and
INTR_INTERLOCK to improve readability. Same object code generated as before.

branches: 1.202.2;
use ENTRY_NOPROFILE() for various entry points,
stack traces work better that way.

branches: 1.201.4;
remove cache_flush_virt() and PADDRT, they're no longer used.
allocate a stack frame for blast_dcache() when profiling so it shows up.
in dcache_flush_page(), use a stride of 32 instead of 16 to match the
cache line size. correct various comments.

Don't compile sparc64_ipi functions when UP.

No need to clear %o1 when returning a 64 bit integer from pseg_get if we
are compiling for a 64 bit kernel.

Change interrupt pending list end value to -1, to avoid linking an
interrupt handler twice.

It may fix "sleep forever" bug.

Clean-up unused defines and variables.

branches: 1.196.2;
Use want_resched from cpu_info area.

Replace intrcnts by evcnts. XXX - needs slight tweaking for MULTIPROCESSOR.

checkpoint of MP work from dennis and myself. includes cross-processor
interrupt framework, a sledgehammer TLB invalidation and misc MP fixes.
doesn't work at all yet.

Uppercase CPU, plural is CPUs.

Rename switchexit to cpu_exit and g/c the old C implementation of cpu_exit.

Support RAS for 32bit kernels too.

In proc_trampoline don't bother to clear condition codes in the trap
frame, since we no longer rely on this.
While there fix a few comments.

Minor cleanup

Implement restartable atomic sequences (RAS) for sparc64.

Spinup secondary cpus. Based on codes sent to me by Dennis Chernoivanov
and Chuck Silvers.

update to recent changes - exit2() doesn't exist anymore and the
exit path always calls lwp_exit2()
pointed out Martin Husemann

Minor cleanup: remove unused code, sync a comment with current reality.

Use per-cpu pcb, curlwp and fplwp rather than global ones. This brings
GENERIC.MP configuration to a usable state.

Approved by petrov@.

Call 'proc_trampoline_mp' from 'proc_trampoline' in a multiprocessor
configuration. Fix process locking. Approved by petrov@.

Remove nwindows calculation, it's not used. Set l_cpu in cpu_switch.

Get rid of bcopy/bzero.

Initial siginfo support for sparc64 (untested). COMPAT_16 sigcontext signal
delivery tested.

add support for non-executable mappings (where the hardware allows this)
and make the stack and heap non-executable by default. the changes
fall into two basic catagories:

- pmap and trap-handler changes. these are all MD:
= alpha: we already track per-page execute permission with the (software)
PG_EXEC bit, so just have the trap handler pay attention to it.
= i386: use a new GDT segment for %cs for processes that have no
executable mappings above a certain threshold (currently the
bottom of the stack). track per-page execute permission with
the last unused PTE bit.
= powerpc/ibm4xx: just use the hardware exec bit.
= powerpc/oea: we already track per-page exec bits, but the hardware only
implements non-exec mappings at the segment level. so track the
number of executable mappings in each segment and turn on the no-exec
segment bit iff the count is 0. adjust the trap handler to deal.
= sparc (sun4m): fix our use of the hardware protection bits.
fix the trap handler to recognize text faults.
= sparc64: split the existing unified TSB into data and instruction TSBs,
and only load TTEs into the appropriate TSB(s) for the permissions.
fix the trap handler to check for execute permission.
= not yet implemented: amd64, hppa, sh5

- changes in all the emulations that put a signal trampoline on the stack.
instead, we now put the trampoline into a uvm_aobj and map that into
the process separately.

originally from openbsd, adapted for netbsd by me.

traceit: don't use %g7 as it is to become curcpu() register. Add comments.

branches: 1.177.2;
Make kernels with options KGDB at least compile.
Still needs some work to make it working.

Use BLOCK_ALIGN rather than BLOCK_SIZE for aligning a pointer.

Preserve the symbol table in the LKM || KSYMS cases also.

use branch slot in pmap_copy_page.

Divide `set' instruction in delay slot (from OpenBSD change to rev 1.30).

Remove some #if 0 code and a few XXX marked membars w/o proper
justification for their existence.

Remove left over pieces from reusing the signal trampoline for upcalls.
This should fix signal delivery for 32bit kernels.

Switch to use cycle counter (%tick) based microtime().

This is derived from alpha/microtime.c and i386/tsc_microtime.c,
and will share with both ports.

This should fix PR port-sparc64/18452.
(approved by martin)

Fix return value of pseg_set, which I accidently broke in rev. 1.164.

Fix register names in comment and shave off another register (hint
from uwe).

Rearrange register usage in pseg_set a bit and shave off %g4.

Remove a few #ifdef _LP64 by using STPTR/LDPTR macros.
Thanks to Matthew Green for the pointer.

Apply leaf procedure optimization to pseg_set (again).

Make the pmap count resident/wired mappings on the fly instead of
walking the page tables whenever this information is needed.

Add an option PMAP_COUNT_DEBUG to assert the new counts and the
page table walk agree.

The old solution had very bad performance impact, for example
by the high CPU load when running top(1).

Thanks to Simon Burge for pointing at the cause of the problem and
to Valeriy E. Ushakov for optimizing my simple minded assembler code.

Remove unused function pseg_find (which looked quite similar to
pseg_get anyway).

Merge the nathanw_sa branch.

Move dump_dtlb back under DDB.

fix two bugs in my last pmap rework:
- in tlb_flush_all(), don't skip TLB entries with the high bit on,
I was confused about which MMU register it was using. it's also fine
to use the last hardware context.
- in pmap_create(), don't allocate a hardware context for the new pmap.
it's unnecessary, and when this would cause us to recycle all the contexts,
it would result in the current process's context being set to 0
(ie. the kernel's context). the current process could then return to
userspace without going through the context-switch code (and thus without
having a hardware context reallocated). this would lead to user mappings
being entered in the kernel's context later, causing all sorts of trouble.
add some assertions to catch this kind of thing.

new function dump_itlb.

many improvements:
- use struct vm_page_md for attaching pv entries to struct vm_page
- change pseg_set()'s return value to indicate whether the spare page
was used as an L2 or L3 PTP.
- use a pool for pv entries instead of malloc().
- put PTPs on a list attached to the pmap so we can free them
more efficiently (by just walking the list) in pmap_destroy().
- use the new pmap_remove_all() interface to avoid flushing the cache and TLB
for each pmap_remove() that's done as we are tearing down an address space.
- in pmap_enter(), handle replacing an existing mapping more efficiently
than just calling pmap_remove() on it. also, skip flushing the
TSB and TLB if there was no previous mapping, since there can't be
anything we need to flush. also, preload the TSB if we're pre-setting
the mod/ref bits.
- allocate hardware contexts like the MIPS pmap:
allocate them all sequentially without reuse, then once we run out
just invalidate all user TLB entries and flush the entire L1 dcache.
- fix pmap_extract() for the case where the va is not page-aligned and
nothing is mapped there.
- fix calculation of TSB size. it was comparing physmem (which is
in units of pages) to constants that only make sense if they are
in units of bytes.
- avoid sleeping in pmap_enter(), instead let the caller do it.
- use pmap_kenter_pa() instead of pmap_enter() where appropriate.
- remove code to handle impossible cases in various functions.
- tweak asm code to pipeline a little better.
- remove many unnecessary spls and membars.
- lots of code cleanup.
- no doubt other stuff that I've forgotten.

the result of all this is that a fork+exit microbenchmark is 34% faster
and a fork+exec+exit microbenchmark is 28% faster.

Spell 'should' correctly.

Shave 10% off the time needed to flush the D$ through a bit of instruction
reordering.

Handle improperly aligned stack pointers more cleanly.

Fix the fork trampoline to properly use the supplied tstate.

Make this work with an ISO C preprocessor.

branches: 1.153.2; 1.153.4;
Only save locals and ins in trapframes for DEBUG kernels.

Add some comments to the CPP conditionals controlling the trap macro
definitions, to ease navigation somewhat.

Make lots of routines profile-friendly and get rid of some cruft.

Not needed debug stuff deleted.

tell newer binutils that our use of %g2 and %g3 is OK here.

Cleanup MMU initialization to prevent hangs.

branches: 1.147.4;
Fix microsecond calculation in microtime.

Fix register allocation in microtime().

Shift the UPAID in the correct direction.

Be more tolerant of unaligned stack pointers.

Enable block insns in bcopy and bzero.

In bcopy use 32-bit comparisons in 32-bit kernels.

Use correct calculation for the trapframe in softtrap for ILP32.

- pull in opt_kgdb.h where necessary
- replace opt_kgdb_machdep.h with opt_kgdb.h
- defparam opt_kgdb.h:
KGDB_DEV KGDB_DEVNAME KGDB_DEVADDR KGDB_DEVRATE KGDB_DEVMODE
- move from opt_ddbparam.h to opt_ddb.h:
DDB_FROMCONSOLE DDB_ONPANIC DDB_HISTORY_SIZE DDB_BREAK_CHAR SYMTAB_SPACE
- replace KGDBDEV with KGDB_DEV
- replace KGDBADDR with KGDB_DEVADDR
- replace KGDBMODE with KGDB_DEVMODE
- replace KGDBRATE with KGDB_DEVRATE
- use `9600' instead of `0x2580' for 9600 baud rate
- use correct quotes for options KGDB_DEVNAME="\"com\""
- use correct quotes for options KGDB_DEV="17*256+0"
- remove unnecessary dependancy on Makefile for kgdb_stub.o
- minor whitespace cleanup

branches: 1.139.4;
Don't declare intrhand[] here -- that's taken care of in intr.c.

branches: 1.138.2;
Preliminary UltraSPARC III suport (not enabled yet).

g/c _remque()/_insque(); use the C version in libkern

Fix context masking lossage caused by last version.

Clean up the trap code to handle the changes cause by supporting non-faulting
accesses. Also change the parameter order so trap frames are easy to find.

Make sure C code always has ASI_PRIMARY_NOFAULT in the %asi register so
the compiler will be happy if it ever starts generating speculative loads.

Add support for non-faulting loads and optimize MMU miss handling paths.

Fix bcopy.

Sync bzero w/libc version.

Integrate fixes from userland memset.

Code cleanup and optimization.

Get rid of some debug code that really isn't all that useful any more.

Do ignore unregistered interrupts instead of faulting.

Make bcopy a 32-bit friendly.

Fix problems booting 32-bit kernels and dispatching level-15 interrupts.

branches: 1.124.2;
Code cleanup. Add block_disable to let you turn on block set/copy routines.

New block bzero and bcopy.

Replace old bcopy with a much faster version.

Disable axidently enabled debug code to make -current compile again.

Use a newer version (still disabled) of block bcopy.

Fixup fpu save restore, get rid of some unused copy functions, and add
a new bcopy.

Allow multiple 4MB pages for both kernel text and kernel data.

Note: DEBUG checks will not allow accesses to the hole between
text and data segments, so you may need to update programs that
use libkvm before relocating the start of kernel data.

extract the netbsd32, sunos, 32 bit svr4 and 64 bit svr4 sigcode out into
separate files, so that compat lkms can link with them as well.

add missing #include "opt_kgdb.h"

Spell occurred correctly.

Interrupt latency fix from sparc port. (Hm. One insn instead of 5).

Fix inverted preprocessor logic. (Spotted by Charles.)

branches: 1.112.2;
Add support for 64-bit SVR4 (Solaris 8) binaries.

Fix conditional for svr4 compat inclusion.

Add signal trampoline for 64-bit svr4 emulations.

Make INCR() atomic.

pseg_get tells us whether it really needed that spare page after all.

Fixup calculating where fpstate should be saved in the signal trampoline
code so it doesn't overwrite something important like the registers.

Fix ILP32 bug in probe_get().

Restore revision 1.102.

I hope it sticks this time..

Fix hang in 32-bit kernels caused by relocation of KERNBASE.

Fix interrupt lossage problem and some cosmetic changes.

cpu_initialize: global registers aren't preserved across calls to
prom_printf(), so replace %g3 with %l7 (which isn't used despite
the `usage' comment).

Also, clean up white-space turds.

This seems to fix the interrupt lossage of non-debug kernels.

Fix INTR_SETUP() to properly distinguish kernel and user addresses.

Make microtime() return the correct time on %tick machines.

Re-enable debug stuff in dostart for now to detect kernel mapping issues.

Missing `#if defined(MULTIPROCESSOR) || defined(LOCKDEBUG)'

Make the improved %tick changes work with machines that use the counter-timer
for the clock.

Fix new version of microtime.

(Hopefully) fix clock skew on machines that us %tick for the clock (UIIi).

Use casx instead of stx to change entries in the page table trees.

Update the scheduler to the new locking scheme.

Fix signal delivery.

Get kernel profiling to work. Now if only gprof worked....

Overhaul cache flush code and coredump code.

Re-instate a couple of delay slot NOPs marked `XXX spitfire bug?'

Optimization to pmap_zero_page(): we only need to use the first 16 FP regs,
as in bzero().

Fix a whitespace glitch.

Pull in the bzero() changes from libc for real, with a couple of kernel-
specific hacks:
* Make the bias handling depend on STKB, so it switches correctly for 32-bit
kernels.
* Use the 32-bit stw/ld for 32-bit kernels.
* Add a few `-STKB's that were missed in the previous change.

Cosmetic & readability:
- local label cleanup in interrupt handling code;
- in debugging sections when testing `intrdebug': use a unique
local label (97) to enhanced clarity;
- comment-out unused code;
- kill a bug as the satisfying result of doing all of the above.

Fix stack issues in bzero.

Revert the previous change for now. I don't have time to debug it tonight.

Copy bzero change from libc.

Since `intrcnt' points to an array of long ints, reserve the appropriate
amount of space depending on _LP64. Provide macros to load/store signed
and unsigned long ints.

Also, make the `intrcnt' and `intrpending' arrays 16 entries long, as on
sparc/v9 interrupt #15 is just a regular interrupt.

Fix merge problem.

Disable interrupt interlock using ih->ih_pending field. Seems to cause
more problems than it solves.

Miscellaneous cleanup.

Correct args usage comment in probeget.
Whitespace cleanup.

probeget: 9 -> 0x9

If `ih_pending' is set simply return from the trap, don't schedule another
soft interrupt.

openfirmware: don't lower %pil

Kill typos in interrupt code.

probeget: do the same song and dance as ld*a() macros in ctlreg.h do.

Move some macro definitions out of genassym and into locore.s

backout part of previous: turn off intrdebug, and another debug message

Changes for readability:
- #define LP64 dependent STACKFRAME macro once
- use a LOAD_ASCIZ macro for setting up the format string
for debug messages.

Temporary fix to prevent the clock from stopping on UIIi machines.

Prevent interrupts from being posted if they are active and deprecate polled
interrupts. This seems to eliminate the spurious interrupts.

Clean up interrupt handling some more.

Fix interrupt delivery on UltraSPARC IIi machines.

With these changes the kernel seems almost stable again.

More VM cleanup:

Move the cpu_info structure above the interrupt stack so it won't
get corrupted if the stack overflows.

Flush the D$ before and after all MMU bypass accesses since the
D$ latches all of those.

Make probeget() and probeset() really work for a change.

branches: 1.62.2;
Have separate data and text segments, make text read-only.

Start reorganizing the kernel for MULTIPROCESSOR support.

Add a comment about needing to initialize p_cpu when multiple
processors are supported.

branches: 1.59.2;
First sweep at scheduler state cleanup. Collect MI scheduler
state into global and per-CPU scheduler state:

- Global state: sched_qs (run queues), sched_whichqs (bitmap
of non-empty run queues), sched_slpque (sleep queues).
NOTE: These may collectively move into a struct schedstate
at some point in the future.

- Per-CPU state, struct schedstate_percpu: spc_runtime
(time process on this CPU started running), spc_flags
(replaces struct proc's p_schedflags), and
spc_curpriority (usrpri of processes on this CPU).

- Every platform must now supply a struct cpu_info and
a curcpu() macro. Simplify existing cpu_info declarations
where appropriate.

- All references to per-CPU scheduler state now made through
curcpu(). NOTE: this will likely be adjusted in the future
after further changes to struct proc are made.

Tested on i386 and Alpha. Changes are mostly mechanical, but apologies
in advance if it doesn't compile on a particular platform.

Don't set `runtime' here; it's done at the end of mi_switch().
cvS: ----------------------------------------------------------------------

Introduce a new process state distinct from SRUN called SONPROC
which indicates that the process is actually running on a
processor. Test against SONPROC as appropriate rather than
combinations of SRUN and curproc. Update all context switch code
to properly set SONPROC when the process becomes the current
process on the CPU.

If we trap due to a bad kernel stack, try to switch to the interrupt strack
before breaking into the debugger.

Prevent the system from getting watchdog reset by returning directly to user
mode if restoring the user window traps. Also reduce the time we run with
%tl>0.

Apparently gas 2.9.5 (don't ask) does not like to add values to %lo()
directives so move the additions inside %lo() to make it happy.
Hopefully this won't break older versions.

- fix a speeling error
- rename some numbered labels to be a bit less confusing.
- remove dead code
- fix a debug error message (argh!)

Get a kernel without DDB to build.

Revamp interrupts again:

Fix a bug causing interrmittent panics in interrupt dispatch.
Use interrupt vectors for softints.
Add a new send_softint interface.
Improved D$ flushing.
Improve traptrace and other debugging enhancements.

Interrupt handlers return zero if they didn't handle the interrupt.

Add debugger single-stepping and fix interrupt dispatch bugs.

Lots of changes:

remove ffs()
add bzero that uses block store insns
save %gsr to fpstate
revert to physical accesses for pmap_zero_page so we don't
have conflicting page mappings
general cleanup.

Handle zero length properly in copyinstr() copyoutstr() and copystr().

branches: 1.46.2; 1.46.4;
Update to post 1.4.

branches: 1.45.2;
Make broken versions of gas happy by using sethi/or instead of set for
(negative) addresses.

Centralize the declaration and clearing of `cold'.

More fun with pmap.

Improve TRAPTRACE. Add FLTRACE that only tracks MMU faults.

Make pbrobeget() and probeset() work for 64-bit values as well. To do this
the arguments are changed so the address is first and the ASI second so we
can have the address in %o0:%o1 and not worry about unused registers.

Also a bit of copyright cleanup.

update a commit slightly.

Add code to support using %tick as the system clock if no "counter-timer" node
is found.

Fixup non-DEBUG kernels.

Add sunos_sigcode (32 bit only)

branches: 1.36.4;
COMPAT_SPARC32 -> COMPAT_NETBSD32

clean and up make compile

completely remove Mach VM support. all that is left is the all the
header files as UVM still uses (most of) these.

Need to set up the pointers properly for partial FPU register stores.

Flush D$ on bypass accesses.

Use block load/store in pmap_zero_page()/pmap_copy_page().

Let's test out vectored interrupts.

RCS ID police

fix a comment pasto.

Finally fixed 64-bit signal trampoline.

Handle the symbol table properly in a 32-bit kernel that was booted from a
64-bit bootloader.

More signal fixup.

Sync kcopy() with 32-bit SPARC version.

Fixup the last broken bits of the signal handling code.

Fix more syscall32 and signal breakage.

More fixup in the signal area.

64-bit syscall cleanup.

Add emulation for some new FPU insns: conversion to 64-bit long int and
conditional moves.

SYS_sigreturn -> SYS___sigreturn14.

Fixup signal changes (hopefully). However SUN_COMPAT is likely broken by
this and I don't know how to fix it.

We can now exec a 64-bit init through a really ugly hack (don't ask.)

Add labels for the compat_sparc32 signal trampoline and break -- er -- fix
suword and fuword.

Don't use %tick for microtime on 32-bit kernels -- breaks gettimeofday.

Adjust for the new "reaper" kernel thread: do not free the vmspace and
u-area in machine-dependent code. Instead, call exit2() to schedule
the reaper to free them for us, once it is safe to do so (i.e. we are
no longer running on the dead proc's vmspace and stack).

Fix data corruption problem caused by broken debug code.

Add new 64-bit trap vectors.

32-bit fixup.

Disable some debug stuff.

It slices. It dices. It does everything except exec a sparc32_compat init.

Periodic update: now starts probing devices.

Some more 64-bit cleanup. Now everything compiles.

Some more 64-bit-ification.

Add 64-bit pmap support:

Moved from a two level 512/1024 entry setup mapping 32 (9/10/13) bits
respectively to a three level 1024/1024/1024 entry setup mapping 43
(10/10/10/13) bits. In 32-bit mode we waste about 1/12 pages mapping the high
11 bits. We also only manage 43 of the possible 44 bits of virtual address
space, wasting half of it. Oh well, maybe we'll do better next revision.

panicstack: is needed outside of DEBUG. use _C_LABEL, and printf not db_printf (incase no ddb!)

Merge paddr_t changes into the main branch.

branches: 1.4.2;
General update:

Added genassym.cf
Removed lderr which should never have gotten in
Removed lots of dead code from locore.s
Added some softint stuff to intr.c
Added support for halt -p
esp and le both use bus_dmamap_*() functions now
instead of kdvma_mapin()
groundwork for PCI (but we still have no drivers for
any sun4u PCI devices)

defopt DDB.

defopt COMPAT_SVR4

branches: 1.1.1;
Initial revision

sparc64: Optimization: Omit needless membar when triggering softint.

When we are triggering a softint, it can't already hold any mutexes.
So any path to mutex_exit(mtx) must go via mutex_enter(mtx), which is
always done with atomic r/m/w, and we need not issue any explicit
barrier between ci->ci_curlwp = softlwp and a potential load of
mtx->mtx_owner in mutex_exit.

PR kern/57240

XXX pullup-8
XXX pullup-9
XXX pullup-10

sparc64: Add missing barriers in cpu_switchto.

Details in comments.

PR kern/57240

XXX pullup-8
XXX pullup-9
XXX pullup-10

fix various typos in comments.

fix various typos in comments.

fix various small typos, mainly in comments.

try to handle ultrasparc III* "fast ecc error" traps.

i had a sunblade 2500 reset with this trap recently, and the
manual says it should have been recoverable. in particular,
the US IIIv2 manual says in section 12.4.4.2 that the i-cache
won't see this data ever (so nothing needed there), and that
the d-cache needs to be disabled, and all the entries flushed,
and then re-enabled.

tested with a "trap 0x70" from kernel space, that trap.c
converted the "0x170" back to "0x70" (with an ADVANCE). this
code not commited.