Stop making <machine/pmap.h> include <machine/pte.h>, and fix the very few
files which really need <machine/pte.h> guts.

Implement per-CPU caching for the page table page (vp) pool and the PTE
descriptor (pted) pool in the arm64 pmap implementation. This
significantly reduces the side-effects of lock contention on the kernel
map lock that is (incorrectly) translated into excessive page daemon
wakeups. This is not a perfect solution but it does lead to significant
speedups on machines with many CPU cores.

This requires adding a new pmap_init_percpu() function that gets called
at the point where kernel is ready to set up the per-CPU pool caches.
Dummy implementations of this function are added for all non-arm64
architectures. Some other architectures can probably benefit from
providing an actual implementation that sets up per-CPU caches for
pmap pools as well.

ok phessler@, claudio@, miod@, patrick@

On systems without xonly mmu hardware-enforcement, we can still mitigate
against classic BROP with a range-checking wrapper in front of copyin() and
copyinstr() which ensures the userland source doesn't overlap the main program
text, ld.so text, signal tramp text (it's mapping is hard to distinguish
so it comes along for the ride), or libc.so text. ld.so tells the kernel
libc.so text range with msyscall(2). The range checking for 2-4 elements is
done without locking (because all 4 ranges are immutable!) and is inexpensive.

write(sock, &open, 400) now fails with EFAULT. No programs have been
discovered which require reading their own text segments with a system call.

On a machine without mmu enforcement, a test program reports the following:
userland kernel
ld.so readable unreadable
mmap xz unreadable unreadable
mmap x readable readable
mmap nrx readable readable
mmap nwx readable readable
mmap xnwx readable readable
main readable unreadable
libc unmapped? readable unreadable
libc mapped readable unreadable

ok kettenis, additional help from miod

Add and use a specific routine to fetch instructions from userland when
doing floating-point emulation, as copyin will fail on exec-only mappings
now.
ok deraadt@ kettenis@

Remove dead/unused prototypes

With the introduction of the PMAP_PREFER_{ALIGN,OFFSET} macros a long time ago,
there are actually no more uses of the PMAP_PREFER() macro left in the kernel.
Remove that macro but keep PMAP_PREFER as a simple #define for it to let uvm
knows the PMAP_PREFER_{ALIGN,OFFSET} macros are available.

ok mpi@

Remove pmap_collect() when a no-op, define __HAVE_PMAP_COLLECT otherwise.
Use that define to shunt uvm_swapout_threads(), which is a noop when
pmap_collect() does nothing.

ok mpi@

Remove old debug code knobs, wrong comments, and unused pv entry flags.
ok kettenis@

Make the sparc64 pmap (more) mpsafe by protecting both the pmap itself and the
pv lists with a mutex. Some minor code adjustments to prevent holding locks
too long. This should make pmap_enter(9), pmap_remove(9) and
pmap_page_protect(9) safe to use without holding the kernel lock.

minimal removal of simplelock to eliminate lock.h dependency

Move declaration of struct vm_page_md from <machine/vmparam.h> to
<machine/pmap.h> where it belongs, and compensate in <uvm/uvm_extern.h>
by including <uvm/uvm_pmap.h> before <uvm/uvm_page.h>. Tested on all
MACHINE_ARCH but amd64 and i386 (and hppa64).

Add PMAP_NOCACHE as an alias for PMAP_NC for use in semi-MI code.

final removal of daddr64_t. daddr_t has been 64 bit for a long enough
test period; i think 3 years ago the last bugs fell out.
ok otto beck others

try to avoid pulling in pte.h and other more crazy things. Checked against
the things that libkvm needs.

refactor sys/param.h and machine/param.h. A lot of #ifdef _KERNEL is added
to keep definitions our of user space. The MD files now follow a consistant
order -- all namespace intrusion is at the tail can be cleaned up
independently. locore, bootblocks, and libkvm still see enough visibility to
build. Checked on 90% of platforms...

Turns out VA_ALIAS_MASK is indeed a mask, but with the PAGE_MASK deceptively
zeroed out.
Documented this in code and updated PMAP_PREFER_* macros to use the corrected
values.

Discussed with and ok miod@

Better make PMAP_PREFER_ALIGN() a rounded value, as done on other arches;
while there fix PMAP_PREFER_OFFSET() for good too, after discussion with ariane@

Off-by-two in PMAP_PREFER_ALIGN definition; found the hard way by matthieu@

Expose pmap_prefer parameters.
Enables future uvm_map code to make intelligent decisions during allocation.

No functional change.

Kill pmap_phys_address(), and force every driver's mmap() routine to return
a physical address [more precisely, something suitable to pass to pmap_enter()'sphysical address argument].

This allows MI drivers to implement mmap() routines without having to know
about the pmap_phys_address() implementation and #ifdef obfuscation.

Change the signature of PMAP_PREFER from void PMAP_PREFER(..., vaddr_t *) to
vaddr_t PMAP_PREFER(..., vaddr_t). This allows better compiler optimization
when the function is inlined, and avoids accessing memory on architectures
when we can pass function arguments in registers.

Add a pm_statistics struct to all pmap, and keep track of resident
pages. Use this to provide a real pmap_resident_count() function.
ok kettenis@

Sparc64 MMUs have an address hole, too, and the pmap implementation we
are using has an even larger one, so implement pmap_remove_hole() to
prevent mmap() from ever reaching the hole.

feedback and ok kettenis@

Remove a lot of symbols from the namespace, otherwise sys/sysctl.h and
rpc/pmap_prot.h collide.. "struct pmap" from the kernel should not make
it out to userland.

Spin up secondary CPUs on MULTIPROCESSOR kernels. Works on UltraSPARC-III
CPUs.

Introduce a md pmap hook, pmap_remove_holes(), which is supposed to mark
the holes a MMU may have from a given vm_map. This will be automagically
invoked for newly created vmspaces.

On platforms with MMU holes (e.g. sun4, sun4c and vax), this prevents
mmap(2) hints which would end up being in the hole to be accepted as valid,
causing unexpected signals when the process tries to access the hole
(since pmap can not fill the hole anyway).

Unfortunately, the logic mmap() uses to pick a valid address for anonymous
mappings needs work, as it will only try to find an address higher than the
hint, which causes all mmap() with a hint in the hole to fail on vax. This
will be improved later.

now that all partition size/offsets are potentially 64-bit, change the
type of all variables to daddr64_t. this includes the APIs for XXsize()
and XXdump(), all range checks inside bio drivers, internal variables
for disklabel handling, and even uvm's swap offsets. re-read numerous
times by otto, miod, krw, thib to look for errors

Fix PMAP_PREFER and make it use VA_ALIAS_MASK to prevent me from breaking
it again.

ok miod@, jason@

It's bit 13 that's causing the address aliasing in the cache, not bit 14.
ok jason@

Nuke unused pmap_from_phys_address().

Needs a non-empty pmap_unuse_final() as sparc.
Spotted by: art@ No cookie for: deraadt@

rename sparc kill_user_windows() to pmap_unuse_final(). provide empty stubs
on all other architectures. remove last architecture dependent #ifdef from
uvm code.

Properly flush instruction cache for ptrace(PT_WRTIE_{DI}, ...) on powerpc
and m68k.
ok drahn@, millert@

Final __P removal plus some cosmetic fixups

First round of __P removal in sys

Implement vgafb_mmap() fully
From NetBSD: change bus_space_mmap() prototype to standard one

branches: 1.2.2;
Yet another sync to NetBSD uvm.
Today we add a pmap argument to pmap_update() and allocate map entries for
kernel_map from kmem_map instead of using the static entries. This should
get rid of MAX_KMAPENT panics. Also some uvm_loan problems are fixed.

branches: 1.1.4;
Some more includes from NetBSD.
No modifications.

Implement per-CPU caching for the page table page (vp) pool and the PTE
descriptor (pted) pool in the arm64 pmap implementation. This
significantly reduces the side-effects of lock contention on the kernel
map lock that is (incorrectly) translated into excessive page daemon
wakeups. This is not a perfect solution but it does lead to significant
speedups on machines with many CPU cores.

This requires adding a new pmap_init_percpu() function that gets called
at the point where kernel is ready to set up the per-CPU pool caches.
Dummy implementations of this function are added for all non-arm64
architectures. Some other architectures can probably benefit from
providing an actual implementation that sets up per-CPU caches for
pmap pools as well.

ok phessler@, claudio@, miod@, patrick@

On systems without xonly mmu hardware-enforcement, we can still mitigate
against classic BROP with a range-checking wrapper in front of copyin() and
copyinstr() which ensures the userland source doesn't overlap the main program
text, ld.so text, signal tramp text (it's mapping is hard to distinguish
so it comes along for the ride), or libc.so text. ld.so tells the kernel
libc.so text range with msyscall(2). The range checking for 2-4 elements is
done without locking (because all 4 ranges are immutable!) and is inexpensive.

write(sock, &open, 400) now fails with EFAULT. No programs have been
discovered which require reading their own text segments with a system call.

On a machine without mmu enforcement, a test program reports the following:
userland kernel
ld.so readable unreadable
mmap xz unreadable unreadable
mmap x readable readable
mmap nrx readable readable
mmap nwx readable readable
mmap xnwx readable readable
main readable unreadable
libc unmapped? readable unreadable
libc mapped readable unreadable

ok kettenis, additional help from miod

Add and use a specific routine to fetch instructions from userland when
doing floating-point emulation, as copyin will fail on exec-only mappings
now.
ok deraadt@ kettenis@

Remove dead/unused prototypes

With the introduction of the PMAP_PREFER_{ALIGN,OFFSET} macros a long time ago,
there are actually no more uses of the PMAP_PREFER() macro left in the kernel.
Remove that macro but keep PMAP_PREFER as a simple #define for it to let uvm
knows the PMAP_PREFER_{ALIGN,OFFSET} macros are available.

ok mpi@

Remove pmap_collect() when a no-op, define __HAVE_PMAP_COLLECT otherwise.
Use that define to shunt uvm_swapout_threads(), which is a noop when
pmap_collect() does nothing.

ok mpi@

Remove old debug code knobs, wrong comments, and unused pv entry flags.
ok kettenis@

Make the sparc64 pmap (more) mpsafe by protecting both the pmap itself and the
pv lists with a mutex. Some minor code adjustments to prevent holding locks
too long. This should make pmap_enter(9), pmap_remove(9) and
pmap_page_protect(9) safe to use without holding the kernel lock.

minimal removal of simplelock to eliminate lock.h dependency

Move declaration of struct vm_page_md from <machine/vmparam.h> to
<machine/pmap.h> where it belongs, and compensate in <uvm/uvm_extern.h>
by including <uvm/uvm_pmap.h> before <uvm/uvm_page.h>. Tested on all
MACHINE_ARCH but amd64 and i386 (and hppa64).

Add PMAP_NOCACHE as an alias for PMAP_NC for use in semi-MI code.

final removal of daddr64_t. daddr_t has been 64 bit for a long enough
test period; i think 3 years ago the last bugs fell out.
ok otto beck others

try to avoid pulling in pte.h and other more crazy things. Checked against
the things that libkvm needs.

refactor sys/param.h and machine/param.h. A lot of #ifdef _KERNEL is added
to keep definitions our of user space. The MD files now follow a consistant
order -- all namespace intrusion is at the tail can be cleaned up
independently. locore, bootblocks, and libkvm still see enough visibility to
build. Checked on 90% of platforms...

Turns out VA_ALIAS_MASK is indeed a mask, but with the PAGE_MASK deceptively
zeroed out.
Documented this in code and updated PMAP_PREFER_* macros to use the corrected
values.

Discussed with and ok miod@

Better make PMAP_PREFER_ALIGN() a rounded value, as done on other arches;
while there fix PMAP_PREFER_OFFSET() for good too, after discussion with ariane@

Off-by-two in PMAP_PREFER_ALIGN definition; found the hard way by matthieu@

Expose pmap_prefer parameters.
Enables future uvm_map code to make intelligent decisions during allocation.

No functional change.

Kill pmap_phys_address(), and force every driver's mmap() routine to return
a physical address [more precisely, something suitable to pass to pmap_enter()'sphysical address argument].

This allows MI drivers to implement mmap() routines without having to know
about the pmap_phys_address() implementation and #ifdef obfuscation.

Change the signature of PMAP_PREFER from void PMAP_PREFER(..., vaddr_t *) to
vaddr_t PMAP_PREFER(..., vaddr_t). This allows better compiler optimization
when the function is inlined, and avoids accessing memory on architectures
when we can pass function arguments in registers.

Add a pm_statistics struct to all pmap, and keep track of resident
pages. Use this to provide a real pmap_resident_count() function.
ok kettenis@

Sparc64 MMUs have an address hole, too, and the pmap implementation we
are using has an even larger one, so implement pmap_remove_hole() to
prevent mmap() from ever reaching the hole.

feedback and ok kettenis@

Remove a lot of symbols from the namespace, otherwise sys/sysctl.h and
rpc/pmap_prot.h collide.. "struct pmap" from the kernel should not make
it out to userland.

Spin up secondary CPUs on MULTIPROCESSOR kernels. Works on UltraSPARC-III
CPUs.

Introduce a md pmap hook, pmap_remove_holes(), which is supposed to mark
the holes a MMU may have from a given vm_map. This will be automagically
invoked for newly created vmspaces.

On platforms with MMU holes (e.g. sun4, sun4c and vax), this prevents
mmap(2) hints which would end up being in the hole to be accepted as valid,
causing unexpected signals when the process tries to access the hole
(since pmap can not fill the hole anyway).

Unfortunately, the logic mmap() uses to pick a valid address for anonymous
mappings needs work, as it will only try to find an address higher than the
hint, which causes all mmap() with a hint in the hole to fail on vax. This
will be improved later.

now that all partition size/offsets are potentially 64-bit, change the
type of all variables to daddr64_t. this includes the APIs for XXsize()
and XXdump(), all range checks inside bio drivers, internal variables
for disklabel handling, and even uvm's swap offsets. re-read numerous
times by otto, miod, krw, thib to look for errors

Fix PMAP_PREFER and make it use VA_ALIAS_MASK to prevent me from breaking
it again.

ok miod@, jason@

It's bit 13 that's causing the address aliasing in the cache, not bit 14.
ok jason@

Nuke unused pmap_from_phys_address().

Needs a non-empty pmap_unuse_final() as sparc.
Spotted by: art@ No cookie for: deraadt@

rename sparc kill_user_windows() to pmap_unuse_final(). provide empty stubs
on all other architectures. remove last architecture dependent #ifdef from
uvm code.

Properly flush instruction cache for ptrace(PT_WRTIE_{DI}, ...) on powerpc
and m68k.
ok drahn@, millert@

Final __P removal plus some cosmetic fixups

First round of __P removal in sys

Implement vgafb_mmap() fully
From NetBSD: change bus_space_mmap() prototype to standard one

branches: 1.2.2;
Yet another sync to NetBSD uvm.
Today we add a pmap argument to pmap_update() and allocate map entries for
kernel_map from kmem_map instead of using the static entries. This should
get rid of MAX_KMAPENT panics. Also some uvm_loan problems are fixed.

branches: 1.1.4;
Some more includes from NetBSD.
No modifications.

On systems without xonly mmu hardware-enforcement, we can still mitigate
against classic BROP with a range-checking wrapper in front of copyin() and
copyinstr() which ensures the userland source doesn't overlap the main program
text, ld.so text, signal tramp text (it's mapping is hard to distinguish
so it comes along for the ride), or libc.so text. ld.so tells the kernel
libc.so text range with msyscall(2). The range checking for 2-4 elements is
done without locking (because all 4 ranges are immutable!) and is inexpensive.

write(sock, &open, 400) now fails with EFAULT. No programs have been
discovered which require reading their own text segments with a system call.

On a machine without mmu enforcement, a test program reports the following:
userland kernel
ld.so readable unreadable
mmap xz unreadable unreadable
mmap x readable readable
mmap nrx readable readable
mmap nwx readable readable
mmap xnwx readable readable
main readable unreadable
libc unmapped? readable unreadable
libc mapped readable unreadable

ok kettenis, additional help from miod

Add and use a specific routine to fetch instructions from userland when
doing floating-point emulation, as copyin will fail on exec-only mappings
now.
ok deraadt@ kettenis@

Remove dead/unused prototypes

With the introduction of the PMAP_PREFER_{ALIGN,OFFSET} macros a long time ago,
there are actually no more uses of the PMAP_PREFER() macro left in the kernel.
Remove that macro but keep PMAP_PREFER as a simple #define for it to let uvm
knows the PMAP_PREFER_{ALIGN,OFFSET} macros are available.

ok mpi@

Remove pmap_collect() when a no-op, define __HAVE_PMAP_COLLECT otherwise.
Use that define to shunt uvm_swapout_threads(), which is a noop when
pmap_collect() does nothing.

ok mpi@

Remove old debug code knobs, wrong comments, and unused pv entry flags.
ok kettenis@

Make the sparc64 pmap (more) mpsafe by protecting both the pmap itself and the
pv lists with a mutex. Some minor code adjustments to prevent holding locks
too long. This should make pmap_enter(9), pmap_remove(9) and
pmap_page_protect(9) safe to use without holding the kernel lock.

minimal removal of simplelock to eliminate lock.h dependency

Move declaration of struct vm_page_md from <machine/vmparam.h> to
<machine/pmap.h> where it belongs, and compensate in <uvm/uvm_extern.h>
by including <uvm/uvm_pmap.h> before <uvm/uvm_page.h>. Tested on all
MACHINE_ARCH but amd64 and i386 (and hppa64).

Add PMAP_NOCACHE as an alias for PMAP_NC for use in semi-MI code.

final removal of daddr64_t. daddr_t has been 64 bit for a long enough
test period; i think 3 years ago the last bugs fell out.
ok otto beck others

try to avoid pulling in pte.h and other more crazy things. Checked against
the things that libkvm needs.

refactor sys/param.h and machine/param.h. A lot of #ifdef _KERNEL is added
to keep definitions our of user space. The MD files now follow a consistant
order -- all namespace intrusion is at the tail can be cleaned up
independently. locore, bootblocks, and libkvm still see enough visibility to
build. Checked on 90% of platforms...

Turns out VA_ALIAS_MASK is indeed a mask, but with the PAGE_MASK deceptively
zeroed out.
Documented this in code and updated PMAP_PREFER_* macros to use the corrected
values.

Discussed with and ok miod@

Better make PMAP_PREFER_ALIGN() a rounded value, as done on other arches;
while there fix PMAP_PREFER_OFFSET() for good too, after discussion with ariane@

Off-by-two in PMAP_PREFER_ALIGN definition; found the hard way by matthieu@

Expose pmap_prefer parameters.
Enables future uvm_map code to make intelligent decisions during allocation.

No functional change.

Kill pmap_phys_address(), and force every driver's mmap() routine to return
a physical address [more precisely, something suitable to pass to pmap_enter()'sphysical address argument].

This allows MI drivers to implement mmap() routines without having to know
about the pmap_phys_address() implementation and #ifdef obfuscation.

Change the signature of PMAP_PREFER from void PMAP_PREFER(..., vaddr_t *) to
vaddr_t PMAP_PREFER(..., vaddr_t). This allows better compiler optimization
when the function is inlined, and avoids accessing memory on architectures
when we can pass function arguments in registers.

Add a pm_statistics struct to all pmap, and keep track of resident
pages. Use this to provide a real pmap_resident_count() function.
ok kettenis@

Sparc64 MMUs have an address hole, too, and the pmap implementation we
are using has an even larger one, so implement pmap_remove_hole() to
prevent mmap() from ever reaching the hole.

feedback and ok kettenis@

Remove a lot of symbols from the namespace, otherwise sys/sysctl.h and
rpc/pmap_prot.h collide.. "struct pmap" from the kernel should not make
it out to userland.

Spin up secondary CPUs on MULTIPROCESSOR kernels. Works on UltraSPARC-III
CPUs.

Introduce a md pmap hook, pmap_remove_holes(), which is supposed to mark
the holes a MMU may have from a given vm_map. This will be automagically
invoked for newly created vmspaces.

On platforms with MMU holes (e.g. sun4, sun4c and vax), this prevents
mmap(2) hints which would end up being in the hole to be accepted as valid,
causing unexpected signals when the process tries to access the hole
(since pmap can not fill the hole anyway).

Unfortunately, the logic mmap() uses to pick a valid address for anonymous
mappings needs work, as it will only try to find an address higher than the
hint, which causes all mmap() with a hint in the hole to fail on vax. This
will be improved later.

now that all partition size/offsets are potentially 64-bit, change the
type of all variables to daddr64_t. this includes the APIs for XXsize()
and XXdump(), all range checks inside bio drivers, internal variables
for disklabel handling, and even uvm's swap offsets. re-read numerous
times by otto, miod, krw, thib to look for errors

Fix PMAP_PREFER and make it use VA_ALIAS_MASK to prevent me from breaking
it again.

ok miod@, jason@

It's bit 13 that's causing the address aliasing in the cache, not bit 14.
ok jason@

Nuke unused pmap_from_phys_address().

Needs a non-empty pmap_unuse_final() as sparc.
Spotted by: art@ No cookie for: deraadt@

rename sparc kill_user_windows() to pmap_unuse_final(). provide empty stubs
on all other architectures. remove last architecture dependent #ifdef from
uvm code.

Properly flush instruction cache for ptrace(PT_WRTIE_{DI}, ...) on powerpc
and m68k.
ok drahn@, millert@

Final __P removal plus some cosmetic fixups

First round of __P removal in sys

Implement vgafb_mmap() fully
From NetBSD: change bus_space_mmap() prototype to standard one

branches: 1.2.2;
Yet another sync to NetBSD uvm.
Today we add a pmap argument to pmap_update() and allocate map entries for
kernel_map from kmem_map instead of using the static entries. This should
get rid of MAX_KMAPENT panics. Also some uvm_loan problems are fixed.

branches: 1.1.4;
Some more includes from NetBSD.
No modifications.

Add and use a specific routine to fetch instructions from userland when
doing floating-point emulation, as copyin will fail on exec-only mappings
now.
ok deraadt@ kettenis@

Remove dead/unused prototypes

With the introduction of the PMAP_PREFER_{ALIGN,OFFSET} macros a long time ago,
there are actually no more uses of the PMAP_PREFER() macro left in the kernel.
Remove that macro but keep PMAP_PREFER as a simple #define for it to let uvm
knows the PMAP_PREFER_{ALIGN,OFFSET} macros are available.

ok mpi@

Remove pmap_collect() when a no-op, define __HAVE_PMAP_COLLECT otherwise.
Use that define to shunt uvm_swapout_threads(), which is a noop when
pmap_collect() does nothing.

ok mpi@

Remove old debug code knobs, wrong comments, and unused pv entry flags.
ok kettenis@

Make the sparc64 pmap (more) mpsafe by protecting both the pmap itself and the
pv lists with a mutex. Some minor code adjustments to prevent holding locks
too long. This should make pmap_enter(9), pmap_remove(9) and
pmap_page_protect(9) safe to use without holding the kernel lock.

minimal removal of simplelock to eliminate lock.h dependency

Move declaration of struct vm_page_md from <machine/vmparam.h> to
<machine/pmap.h> where it belongs, and compensate in <uvm/uvm_extern.h>
by including <uvm/uvm_pmap.h> before <uvm/uvm_page.h>. Tested on all
MACHINE_ARCH but amd64 and i386 (and hppa64).

Add PMAP_NOCACHE as an alias for PMAP_NC for use in semi-MI code.

final removal of daddr64_t. daddr_t has been 64 bit for a long enough
test period; i think 3 years ago the last bugs fell out.
ok otto beck others

try to avoid pulling in pte.h and other more crazy things. Checked against
the things that libkvm needs.

refactor sys/param.h and machine/param.h. A lot of #ifdef _KERNEL is added
to keep definitions our of user space. The MD files now follow a consistant
order -- all namespace intrusion is at the tail can be cleaned up
independently. locore, bootblocks, and libkvm still see enough visibility to
build. Checked on 90% of platforms...

Turns out VA_ALIAS_MASK is indeed a mask, but with the PAGE_MASK deceptively
zeroed out.
Documented this in code and updated PMAP_PREFER_* macros to use the corrected
values.

Discussed with and ok miod@

Better make PMAP_PREFER_ALIGN() a rounded value, as done on other arches;
while there fix PMAP_PREFER_OFFSET() for good too, after discussion with ariane@

Off-by-two in PMAP_PREFER_ALIGN definition; found the hard way by matthieu@

Expose pmap_prefer parameters.
Enables future uvm_map code to make intelligent decisions during allocation.

No functional change.

Kill pmap_phys_address(), and force every driver's mmap() routine to return
a physical address [more precisely, something suitable to pass to pmap_enter()'sphysical address argument].

This allows MI drivers to implement mmap() routines without having to know
about the pmap_phys_address() implementation and #ifdef obfuscation.

Change the signature of PMAP_PREFER from void PMAP_PREFER(..., vaddr_t *) to
vaddr_t PMAP_PREFER(..., vaddr_t). This allows better compiler optimization
when the function is inlined, and avoids accessing memory on architectures
when we can pass function arguments in registers.

Add a pm_statistics struct to all pmap, and keep track of resident
pages. Use this to provide a real pmap_resident_count() function.
ok kettenis@

Sparc64 MMUs have an address hole, too, and the pmap implementation we
are using has an even larger one, so implement pmap_remove_hole() to
prevent mmap() from ever reaching the hole.

feedback and ok kettenis@

Remove a lot of symbols from the namespace, otherwise sys/sysctl.h and
rpc/pmap_prot.h collide.. "struct pmap" from the kernel should not make
it out to userland.

Spin up secondary CPUs on MULTIPROCESSOR kernels. Works on UltraSPARC-III
CPUs.

Introduce a md pmap hook, pmap_remove_holes(), which is supposed to mark
the holes a MMU may have from a given vm_map. This will be automagically
invoked for newly created vmspaces.

On platforms with MMU holes (e.g. sun4, sun4c and vax), this prevents
mmap(2) hints which would end up being in the hole to be accepted as valid,
causing unexpected signals when the process tries to access the hole
(since pmap can not fill the hole anyway).

Unfortunately, the logic mmap() uses to pick a valid address for anonymous
mappings needs work, as it will only try to find an address higher than the
hint, which causes all mmap() with a hint in the hole to fail on vax. This
will be improved later.

now that all partition size/offsets are potentially 64-bit, change the
type of all variables to daddr64_t. this includes the APIs for XXsize()
and XXdump(), all range checks inside bio drivers, internal variables
for disklabel handling, and even uvm's swap offsets. re-read numerous
times by otto, miod, krw, thib to look for errors

Fix PMAP_PREFER and make it use VA_ALIAS_MASK to prevent me from breaking
it again.

ok miod@, jason@

It's bit 13 that's causing the address aliasing in the cache, not bit 14.
ok jason@

Nuke unused pmap_from_phys_address().

Needs a non-empty pmap_unuse_final() as sparc.
Spotted by: art@ No cookie for: deraadt@

rename sparc kill_user_windows() to pmap_unuse_final(). provide empty stubs
on all other architectures. remove last architecture dependent #ifdef from
uvm code.

Properly flush instruction cache for ptrace(PT_WRTIE_{DI}, ...) on powerpc
and m68k.
ok drahn@, millert@

Final __P removal plus some cosmetic fixups

First round of __P removal in sys

Implement vgafb_mmap() fully
From NetBSD: change bus_space_mmap() prototype to standard one

branches: 1.2.2;
Yet another sync to NetBSD uvm.
Today we add a pmap argument to pmap_update() and allocate map entries for
kernel_map from kmem_map instead of using the static entries. This should
get rid of MAX_KMAPENT panics. Also some uvm_loan problems are fixed.

branches: 1.1.4;
Some more includes from NetBSD.
No modifications.

Remove dead/unused prototypes

With the introduction of the PMAP_PREFER_{ALIGN,OFFSET} macros a long time ago,
there are actually no more uses of the PMAP_PREFER() macro left in the kernel.
Remove that macro but keep PMAP_PREFER as a simple #define for it to let uvm
knows the PMAP_PREFER_{ALIGN,OFFSET} macros are available.

ok mpi@

Remove pmap_collect() when a no-op, define __HAVE_PMAP_COLLECT otherwise.
Use that define to shunt uvm_swapout_threads(), which is a noop when
pmap_collect() does nothing.

ok mpi@

Remove old debug code knobs, wrong comments, and unused pv entry flags.
ok kettenis@

Make the sparc64 pmap (more) mpsafe by protecting both the pmap itself and the
pv lists with a mutex. Some minor code adjustments to prevent holding locks
too long. This should make pmap_enter(9), pmap_remove(9) and
pmap_page_protect(9) safe to use without holding the kernel lock.

minimal removal of simplelock to eliminate lock.h dependency

Move declaration of struct vm_page_md from <machine/vmparam.h> to
<machine/pmap.h> where it belongs, and compensate in <uvm/uvm_extern.h>
by including <uvm/uvm_pmap.h> before <uvm/uvm_page.h>. Tested on all
MACHINE_ARCH but amd64 and i386 (and hppa64).

Add PMAP_NOCACHE as an alias for PMAP_NC for use in semi-MI code.

final removal of daddr64_t. daddr_t has been 64 bit for a long enough
test period; i think 3 years ago the last bugs fell out.
ok otto beck others

try to avoid pulling in pte.h and other more crazy things. Checked against
the things that libkvm needs.

refactor sys/param.h and machine/param.h. A lot of #ifdef _KERNEL is added
to keep definitions our of user space. The MD files now follow a consistant
order -- all namespace intrusion is at the tail can be cleaned up
independently. locore, bootblocks, and libkvm still see enough visibility to
build. Checked on 90% of platforms...

Turns out VA_ALIAS_MASK is indeed a mask, but with the PAGE_MASK deceptively
zeroed out.
Documented this in code and updated PMAP_PREFER_* macros to use the corrected
values.

Discussed with and ok miod@

Better make PMAP_PREFER_ALIGN() a rounded value, as done on other arches;
while there fix PMAP_PREFER_OFFSET() for good too, after discussion with ariane@

Off-by-two in PMAP_PREFER_ALIGN definition; found the hard way by matthieu@

Expose pmap_prefer parameters.
Enables future uvm_map code to make intelligent decisions during allocation.

No functional change.

Kill pmap_phys_address(), and force every driver's mmap() routine to return
a physical address [more precisely, something suitable to pass to pmap_enter()'sphysical address argument].

This allows MI drivers to implement mmap() routines without having to know
about the pmap_phys_address() implementation and #ifdef obfuscation.

Change the signature of PMAP_PREFER from void PMAP_PREFER(..., vaddr_t *) to
vaddr_t PMAP_PREFER(..., vaddr_t). This allows better compiler optimization
when the function is inlined, and avoids accessing memory on architectures
when we can pass function arguments in registers.

Add a pm_statistics struct to all pmap, and keep track of resident
pages. Use this to provide a real pmap_resident_count() function.
ok kettenis@

Sparc64 MMUs have an address hole, too, and the pmap implementation we
are using has an even larger one, so implement pmap_remove_hole() to
prevent mmap() from ever reaching the hole.

feedback and ok kettenis@

Remove a lot of symbols from the namespace, otherwise sys/sysctl.h and
rpc/pmap_prot.h collide.. "struct pmap" from the kernel should not make
it out to userland.

Spin up secondary CPUs on MULTIPROCESSOR kernels. Works on UltraSPARC-III
CPUs.

Introduce a md pmap hook, pmap_remove_holes(), which is supposed to mark
the holes a MMU may have from a given vm_map. This will be automagically
invoked for newly created vmspaces.

On platforms with MMU holes (e.g. sun4, sun4c and vax), this prevents
mmap(2) hints which would end up being in the hole to be accepted as valid,
causing unexpected signals when the process tries to access the hole
(since pmap can not fill the hole anyway).

Unfortunately, the logic mmap() uses to pick a valid address for anonymous
mappings needs work, as it will only try to find an address higher than the
hint, which causes all mmap() with a hint in the hole to fail on vax. This
will be improved later.

now that all partition size/offsets are potentially 64-bit, change the
type of all variables to daddr64_t. this includes the APIs for XXsize()
and XXdump(), all range checks inside bio drivers, internal variables
for disklabel handling, and even uvm's swap offsets. re-read numerous
times by otto, miod, krw, thib to look for errors

Fix PMAP_PREFER and make it use VA_ALIAS_MASK to prevent me from breaking
it again.

ok miod@, jason@

It's bit 13 that's causing the address aliasing in the cache, not bit 14.
ok jason@

Nuke unused pmap_from_phys_address().

Needs a non-empty pmap_unuse_final() as sparc.
Spotted by: art@ No cookie for: deraadt@

rename sparc kill_user_windows() to pmap_unuse_final(). provide empty stubs
on all other architectures. remove last architecture dependent #ifdef from
uvm code.

Properly flush instruction cache for ptrace(PT_WRTIE_{DI}, ...) on powerpc
and m68k.
ok drahn@, millert@

Final __P removal plus some cosmetic fixups

First round of __P removal in sys

Implement vgafb_mmap() fully
From NetBSD: change bus_space_mmap() prototype to standard one

branches: 1.2.2;
Yet another sync to NetBSD uvm.
Today we add a pmap argument to pmap_update() and allocate map entries for
kernel_map from kmem_map instead of using the static entries. This should
get rid of MAX_KMAPENT panics. Also some uvm_loan problems are fixed.

branches: 1.1.4;
Some more includes from NetBSD.
No modifications.

With the introduction of the PMAP_PREFER_{ALIGN,OFFSET} macros a long time ago,
there are actually no more uses of the PMAP_PREFER() macro left in the kernel.
Remove that macro but keep PMAP_PREFER as a simple #define for it to let uvm
knows the PMAP_PREFER_{ALIGN,OFFSET} macros are available.

ok mpi@

Remove pmap_collect() when a no-op, define __HAVE_PMAP_COLLECT otherwise.
Use that define to shunt uvm_swapout_threads(), which is a noop when
pmap_collect() does nothing.

ok mpi@

Remove old debug code knobs, wrong comments, and unused pv entry flags.
ok kettenis@

Make the sparc64 pmap (more) mpsafe by protecting both the pmap itself and the
pv lists with a mutex. Some minor code adjustments to prevent holding locks
too long. This should make pmap_enter(9), pmap_remove(9) and
pmap_page_protect(9) safe to use without holding the kernel lock.

minimal removal of simplelock to eliminate lock.h dependency

Move declaration of struct vm_page_md from <machine/vmparam.h> to
<machine/pmap.h> where it belongs, and compensate in <uvm/uvm_extern.h>
by including <uvm/uvm_pmap.h> before <uvm/uvm_page.h>. Tested on all
MACHINE_ARCH but amd64 and i386 (and hppa64).

Add PMAP_NOCACHE as an alias for PMAP_NC for use in semi-MI code.

final removal of daddr64_t. daddr_t has been 64 bit for a long enough
test period; i think 3 years ago the last bugs fell out.
ok otto beck others

try to avoid pulling in pte.h and other more crazy things. Checked against
the things that libkvm needs.

refactor sys/param.h and machine/param.h. A lot of #ifdef _KERNEL is added
to keep definitions our of user space. The MD files now follow a consistant
order -- all namespace intrusion is at the tail can be cleaned up
independently. locore, bootblocks, and libkvm still see enough visibility to
build. Checked on 90% of platforms...

Turns out VA_ALIAS_MASK is indeed a mask, but with the PAGE_MASK deceptively
zeroed out.
Documented this in code and updated PMAP_PREFER_* macros to use the corrected
values.

Discussed with and ok miod@

Better make PMAP_PREFER_ALIGN() a rounded value, as done on other arches;
while there fix PMAP_PREFER_OFFSET() for good too, after discussion with ariane@

Off-by-two in PMAP_PREFER_ALIGN definition; found the hard way by matthieu@

Expose pmap_prefer parameters.
Enables future uvm_map code to make intelligent decisions during allocation.

No functional change.

Kill pmap_phys_address(), and force every driver's mmap() routine to return
a physical address [more precisely, something suitable to pass to pmap_enter()'sphysical address argument].

This allows MI drivers to implement mmap() routines without having to know
about the pmap_phys_address() implementation and #ifdef obfuscation.

Change the signature of PMAP_PREFER from void PMAP_PREFER(..., vaddr_t *) to
vaddr_t PMAP_PREFER(..., vaddr_t). This allows better compiler optimization
when the function is inlined, and avoids accessing memory on architectures
when we can pass function arguments in registers.

Add a pm_statistics struct to all pmap, and keep track of resident
pages. Use this to provide a real pmap_resident_count() function.
ok kettenis@

Sparc64 MMUs have an address hole, too, and the pmap implementation we
are using has an even larger one, so implement pmap_remove_hole() to
prevent mmap() from ever reaching the hole.

feedback and ok kettenis@

Remove a lot of symbols from the namespace, otherwise sys/sysctl.h and
rpc/pmap_prot.h collide.. "struct pmap" from the kernel should not make
it out to userland.

Spin up secondary CPUs on MULTIPROCESSOR kernels. Works on UltraSPARC-III
CPUs.

Introduce a md pmap hook, pmap_remove_holes(), which is supposed to mark
the holes a MMU may have from a given vm_map. This will be automagically
invoked for newly created vmspaces.

On platforms with MMU holes (e.g. sun4, sun4c and vax), this prevents
mmap(2) hints which would end up being in the hole to be accepted as valid,
causing unexpected signals when the process tries to access the hole
(since pmap can not fill the hole anyway).

Unfortunately, the logic mmap() uses to pick a valid address for anonymous
mappings needs work, as it will only try to find an address higher than the
hint, which causes all mmap() with a hint in the hole to fail on vax. This
will be improved later.

now that all partition size/offsets are potentially 64-bit, change the
type of all variables to daddr64_t. this includes the APIs for XXsize()
and XXdump(), all range checks inside bio drivers, internal variables
for disklabel handling, and even uvm's swap offsets. re-read numerous
times by otto, miod, krw, thib to look for errors

Fix PMAP_PREFER and make it use VA_ALIAS_MASK to prevent me from breaking
it again.

ok miod@, jason@

It's bit 13 that's causing the address aliasing in the cache, not bit 14.
ok jason@

Nuke unused pmap_from_phys_address().

Needs a non-empty pmap_unuse_final() as sparc.
Spotted by: art@ No cookie for: deraadt@

rename sparc kill_user_windows() to pmap_unuse_final(). provide empty stubs
on all other architectures. remove last architecture dependent #ifdef from
uvm code.

Properly flush instruction cache for ptrace(PT_WRTIE_{DI}, ...) on powerpc
and m68k.
ok drahn@, millert@

Final __P removal plus some cosmetic fixups

First round of __P removal in sys

Implement vgafb_mmap() fully
From NetBSD: change bus_space_mmap() prototype to standard one

branches: 1.2.2;
Yet another sync to NetBSD uvm.
Today we add a pmap argument to pmap_update() and allocate map entries for
kernel_map from kmem_map instead of using the static entries. This should
get rid of MAX_KMAPENT panics. Also some uvm_loan problems are fixed.

branches: 1.1.4;
Some more includes from NetBSD.
No modifications.