Add MIPS64r2 TLB read inhibit support

OK deraadt@ miod@

Retire OpenBSD/sgi.

OK deraadt@

Guard against repeated inclusion of <mips64/pte.h>. This fixes drm build
on loongson.

Do not use MIPS64r2 instructions on Loongson 2. Now, a generic loongson
kernel has to learn a new trick in order to use TLB RI/XI on 3A2000.

Define PTE_* macros in one place. Use the lwu instruction for 32-bit PTE
loads on all systems as the TLB code does not need sign extension.

Utilize the TLB Execute-Inhibit bit with non-executable mappings on CPUs
that support the Execute-Inhibit exception. This makes user space W^X
effective on Octeon Plus and later Octeon versions.

Feedback from miod@, thanks!
No objection from deraadt@

Shuffle tlblo-related defines, to better show which bits are software only,
and which bits end up in the actual tlb registers.

On non-R8000 kernels, shrink the actual physical address bits to add a new
software bit, PG_SP (for `special'), which will be used shortly. This halves
the physical memory addressable by non-MIPS_PTE64 kernels, which should not
be a problem anyway.

Allow page table entries to be either 32 bits wide (the existing state of the
code), or 64 bits wide, if option MIPS_PTE64.
64-bit ptes allow for physical memory beyond 16GB (34 bits) to be addressable
by the pmap code.

PGSHIFT -> PAGE_SHIFT

Basic R8000 processor support. R8000 processors require MMU-specific code,
exception-specific code, clock-specific code, and L1 cache-specific code. L2
cache is per-design, of which only two exist: SGI Power Indigo2 (IP26) and SGI
Power Challenge (IP21) and are not covered by this commit.

R8000 processors also are 64-bit only processors with 64-bit coprocessor 0
registers, and lack so-called ``compatibility'' memory spaces allowing 32-bit
code to run with sign-extended addresses and registers.

The intrusive changes are covered by #ifdef CPU_R8000 stanzas. However,
trap() is split into a high-level wrapper and a new function, itsa(),
responsible for the actual trap servicing (which name couldn't be helped
because I'm an incorrigible punster). While an R8000 exception may cause
(via trap() ) multiple exceptions to be serviced, non-R8000 processors will
always service one exception in trap(), but they are nevertheless affected
by this code split.

Kill the mostly unused VMTLB_xxx and VMNUM_xxx defines. Move all tlb
knowledge to <machine/pte.h>. Add specific routines for tlb handling setup
(at cpu initialization time) and tlb ASID wrap.

Add support for wired mappings, using the last unused bit in the PTE.

Introduce a #define for the number of PFN bits in a pte, to be used in the
.S code when masking the upper bits, instead of hardcoding them. Makes code
easier to understand (and also I might have a need to reduce PFN width on
some particular CPU models in the future).
No change in generated code.

repair a 32-bit truncation in paddr_t calculation, allowing my 8GB
o350 to boot once again
ok miod

Support for 16KB page size kernels; page size is now set in <machine/param.h>
rather than <mips64/param.h>.

For now, kernels are kept at 4KB to give people some time to build 16KB
compatible binaries; this will change before the end of this release cycle.

Use of 16KB page size kernels yields a 18% speedup (which, offset by the
1.6% slowdown caused by the pmap changes, yields a 16.6% overall speedup).

Use a pool to manage pmap pte pages and top level segment table, instead of
directly allocating pages from uvm; this will allow us to eventually use
a different kernel page size without having to alter the pmap structures
layout.
No functional change; measured slowdown of 1.6% for 4KB page kernels.

Stricter type usage (width and signedness); first step towards 64 bit ptes.

Define the pte cacheability bits from the cpu.h cache coherency constants.

Get rid of the silly union for mips pte. No functional change except
pmap.h now includes pte.h.

Remove unused macros, and make sure the remaining one do not truncate the
high 32 bits of addresses.

More pmap changes from the potpourri pool:
- use wm_page pg_flags pmap-reserved flags to store attributes, instead of
defining a vm_page_md field.
- use atomic operations to touch the above mentioned flags.
- never create ptes with PG_RO and PG_M set (this was harmless anyway).
- in pmap_clear_modify(), do not flush cache if the page was mapped uncached.

Tested on r5k, rm5200, r10k and r12k.

Remove advertising clause from UCB licenses; ok deraad@

Fix some 64 bit address problems.
Some function names made more unique.
Other changes for the upcoming Origin 200 support.

Move <machine/pte.h> to mips64 land.

Retire OpenBSD/sgi.

OK deraadt@

Guard against repeated inclusion of <mips64/pte.h>. This fixes drm build
on loongson.

Do not use MIPS64r2 instructions on Loongson 2. Now, a generic loongson
kernel has to learn a new trick in order to use TLB RI/XI on 3A2000.

Define PTE_* macros in one place. Use the lwu instruction for 32-bit PTE
loads on all systems as the TLB code does not need sign extension.

Utilize the TLB Execute-Inhibit bit with non-executable mappings on CPUs
that support the Execute-Inhibit exception. This makes user space W^X
effective on Octeon Plus and later Octeon versions.

Feedback from miod@, thanks!
No objection from deraadt@

Shuffle tlblo-related defines, to better show which bits are software only,
and which bits end up in the actual tlb registers.

On non-R8000 kernels, shrink the actual physical address bits to add a new
software bit, PG_SP (for `special'), which will be used shortly. This halves
the physical memory addressable by non-MIPS_PTE64 kernels, which should not
be a problem anyway.

Allow page table entries to be either 32 bits wide (the existing state of the
code), or 64 bits wide, if option MIPS_PTE64.
64-bit ptes allow for physical memory beyond 16GB (34 bits) to be addressable
by the pmap code.

PGSHIFT -> PAGE_SHIFT

Basic R8000 processor support. R8000 processors require MMU-specific code,
exception-specific code, clock-specific code, and L1 cache-specific code. L2
cache is per-design, of which only two exist: SGI Power Indigo2 (IP26) and SGI
Power Challenge (IP21) and are not covered by this commit.

R8000 processors also are 64-bit only processors with 64-bit coprocessor 0
registers, and lack so-called ``compatibility'' memory spaces allowing 32-bit
code to run with sign-extended addresses and registers.

The intrusive changes are covered by #ifdef CPU_R8000 stanzas. However,
trap() is split into a high-level wrapper and a new function, itsa(),
responsible for the actual trap servicing (which name couldn't be helped
because I'm an incorrigible punster). While an R8000 exception may cause
(via trap() ) multiple exceptions to be serviced, non-R8000 processors will
always service one exception in trap(), but they are nevertheless affected
by this code split.

Kill the mostly unused VMTLB_xxx and VMNUM_xxx defines. Move all tlb
knowledge to <machine/pte.h>. Add specific routines for tlb handling setup
(at cpu initialization time) and tlb ASID wrap.

Add support for wired mappings, using the last unused bit in the PTE.

Introduce a #define for the number of PFN bits in a pte, to be used in the
.S code when masking the upper bits, instead of hardcoding them. Makes code
easier to understand (and also I might have a need to reduce PFN width on
some particular CPU models in the future).
No change in generated code.

repair a 32-bit truncation in paddr_t calculation, allowing my 8GB
o350 to boot once again
ok miod

Support for 16KB page size kernels; page size is now set in <machine/param.h>
rather than <mips64/param.h>.

For now, kernels are kept at 4KB to give people some time to build 16KB
compatible binaries; this will change before the end of this release cycle.

Use of 16KB page size kernels yields a 18% speedup (which, offset by the
1.6% slowdown caused by the pmap changes, yields a 16.6% overall speedup).

Use a pool to manage pmap pte pages and top level segment table, instead of
directly allocating pages from uvm; this will allow us to eventually use
a different kernel page size without having to alter the pmap structures
layout.
No functional change; measured slowdown of 1.6% for 4KB page kernels.

Stricter type usage (width and signedness); first step towards 64 bit ptes.

Define the pte cacheability bits from the cpu.h cache coherency constants.

Get rid of the silly union for mips pte. No functional change except
pmap.h now includes pte.h.

Remove unused macros, and make sure the remaining one do not truncate the
high 32 bits of addresses.

More pmap changes from the potpourri pool:
- use wm_page pg_flags pmap-reserved flags to store attributes, instead of
defining a vm_page_md field.
- use atomic operations to touch the above mentioned flags.
- never create ptes with PG_RO and PG_M set (this was harmless anyway).
- in pmap_clear_modify(), do not flush cache if the page was mapped uncached.

Tested on r5k, rm5200, r10k and r12k.

Remove advertising clause from UCB licenses; ok deraad@

Fix some 64 bit address problems.
Some function names made more unique.
Other changes for the upcoming Origin 200 support.

Move <machine/pte.h> to mips64 land.

Guard against repeated inclusion of <mips64/pte.h>. This fixes drm build
on loongson.

Do not use MIPS64r2 instructions on Loongson 2. Now, a generic loongson
kernel has to learn a new trick in order to use TLB RI/XI on 3A2000.

Define PTE_* macros in one place. Use the lwu instruction for 32-bit PTE
loads on all systems as the TLB code does not need sign extension.

Utilize the TLB Execute-Inhibit bit with non-executable mappings on CPUs
that support the Execute-Inhibit exception. This makes user space W^X
effective on Octeon Plus and later Octeon versions.

Feedback from miod@, thanks!
No objection from deraadt@

Shuffle tlblo-related defines, to better show which bits are software only,
and which bits end up in the actual tlb registers.

On non-R8000 kernels, shrink the actual physical address bits to add a new
software bit, PG_SP (for `special'), which will be used shortly. This halves
the physical memory addressable by non-MIPS_PTE64 kernels, which should not
be a problem anyway.

Allow page table entries to be either 32 bits wide (the existing state of the
code), or 64 bits wide, if option MIPS_PTE64.
64-bit ptes allow for physical memory beyond 16GB (34 bits) to be addressable
by the pmap code.

PGSHIFT -> PAGE_SHIFT

Basic R8000 processor support. R8000 processors require MMU-specific code,
exception-specific code, clock-specific code, and L1 cache-specific code. L2
cache is per-design, of which only two exist: SGI Power Indigo2 (IP26) and SGI
Power Challenge (IP21) and are not covered by this commit.

R8000 processors also are 64-bit only processors with 64-bit coprocessor 0
registers, and lack so-called ``compatibility'' memory spaces allowing 32-bit
code to run with sign-extended addresses and registers.

The intrusive changes are covered by #ifdef CPU_R8000 stanzas. However,
trap() is split into a high-level wrapper and a new function, itsa(),
responsible for the actual trap servicing (which name couldn't be helped
because I'm an incorrigible punster). While an R8000 exception may cause
(via trap() ) multiple exceptions to be serviced, non-R8000 processors will
always service one exception in trap(), but they are nevertheless affected
by this code split.

Kill the mostly unused VMTLB_xxx and VMNUM_xxx defines. Move all tlb
knowledge to <machine/pte.h>. Add specific routines for tlb handling setup
(at cpu initialization time) and tlb ASID wrap.

Add support for wired mappings, using the last unused bit in the PTE.

Introduce a #define for the number of PFN bits in a pte, to be used in the
.S code when masking the upper bits, instead of hardcoding them. Makes code
easier to understand (and also I might have a need to reduce PFN width on
some particular CPU models in the future).
No change in generated code.

repair a 32-bit truncation in paddr_t calculation, allowing my 8GB
o350 to boot once again
ok miod

Support for 16KB page size kernels; page size is now set in <machine/param.h>
rather than <mips64/param.h>.

For now, kernels are kept at 4KB to give people some time to build 16KB
compatible binaries; this will change before the end of this release cycle.

Use of 16KB page size kernels yields a 18% speedup (which, offset by the
1.6% slowdown caused by the pmap changes, yields a 16.6% overall speedup).

Use a pool to manage pmap pte pages and top level segment table, instead of
directly allocating pages from uvm; this will allow us to eventually use
a different kernel page size without having to alter the pmap structures
layout.
No functional change; measured slowdown of 1.6% for 4KB page kernels.

Stricter type usage (width and signedness); first step towards 64 bit ptes.

Define the pte cacheability bits from the cpu.h cache coherency constants.

Get rid of the silly union for mips pte. No functional change except
pmap.h now includes pte.h.

Remove unused macros, and make sure the remaining one do not truncate the
high 32 bits of addresses.

More pmap changes from the potpourri pool:
- use wm_page pg_flags pmap-reserved flags to store attributes, instead of
defining a vm_page_md field.
- use atomic operations to touch the above mentioned flags.
- never create ptes with PG_RO and PG_M set (this was harmless anyway).
- in pmap_clear_modify(), do not flush cache if the page was mapped uncached.

Tested on r5k, rm5200, r10k and r12k.

Remove advertising clause from UCB licenses; ok deraad@

Fix some 64 bit address problems.
Some function names made more unique.
Other changes for the upcoming Origin 200 support.

Move <machine/pte.h> to mips64 land.

Do not use MIPS64r2 instructions on Loongson 2. Now, a generic loongson
kernel has to learn a new trick in order to use TLB RI/XI on 3A2000.

Define PTE_* macros in one place. Use the lwu instruction for 32-bit PTE
loads on all systems as the TLB code does not need sign extension.

Utilize the TLB Execute-Inhibit bit with non-executable mappings on CPUs
that support the Execute-Inhibit exception. This makes user space W^X
effective on Octeon Plus and later Octeon versions.

Feedback from miod@, thanks!
No objection from deraadt@

Shuffle tlblo-related defines, to better show which bits are software only,
and which bits end up in the actual tlb registers.

On non-R8000 kernels, shrink the actual physical address bits to add a new
software bit, PG_SP (for `special'), which will be used shortly. This halves
the physical memory addressable by non-MIPS_PTE64 kernels, which should not
be a problem anyway.

Allow page table entries to be either 32 bits wide (the existing state of the
code), or 64 bits wide, if option MIPS_PTE64.
64-bit ptes allow for physical memory beyond 16GB (34 bits) to be addressable
by the pmap code.

PGSHIFT -> PAGE_SHIFT

Basic R8000 processor support. R8000 processors require MMU-specific code,
exception-specific code, clock-specific code, and L1 cache-specific code. L2
cache is per-design, of which only two exist: SGI Power Indigo2 (IP26) and SGI
Power Challenge (IP21) and are not covered by this commit.

R8000 processors also are 64-bit only processors with 64-bit coprocessor 0
registers, and lack so-called ``compatibility'' memory spaces allowing 32-bit
code to run with sign-extended addresses and registers.

The intrusive changes are covered by #ifdef CPU_R8000 stanzas. However,
trap() is split into a high-level wrapper and a new function, itsa(),
responsible for the actual trap servicing (which name couldn't be helped
because I'm an incorrigible punster). While an R8000 exception may cause
(via trap() ) multiple exceptions to be serviced, non-R8000 processors will
always service one exception in trap(), but they are nevertheless affected
by this code split.

Kill the mostly unused VMTLB_xxx and VMNUM_xxx defines. Move all tlb
knowledge to <machine/pte.h>. Add specific routines for tlb handling setup
(at cpu initialization time) and tlb ASID wrap.

Add support for wired mappings, using the last unused bit in the PTE.

Introduce a #define for the number of PFN bits in a pte, to be used in the
.S code when masking the upper bits, instead of hardcoding them. Makes code
easier to understand (and also I might have a need to reduce PFN width on
some particular CPU models in the future).
No change in generated code.

repair a 32-bit truncation in paddr_t calculation, allowing my 8GB
o350 to boot once again
ok miod

Support for 16KB page size kernels; page size is now set in <machine/param.h>
rather than <mips64/param.h>.

For now, kernels are kept at 4KB to give people some time to build 16KB
compatible binaries; this will change before the end of this release cycle.

Use of 16KB page size kernels yields a 18% speedup (which, offset by the
1.6% slowdown caused by the pmap changes, yields a 16.6% overall speedup).

Use a pool to manage pmap pte pages and top level segment table, instead of
directly allocating pages from uvm; this will allow us to eventually use
a different kernel page size without having to alter the pmap structures
layout.
No functional change; measured slowdown of 1.6% for 4KB page kernels.

Stricter type usage (width and signedness); first step towards 64 bit ptes.

Define the pte cacheability bits from the cpu.h cache coherency constants.

Get rid of the silly union for mips pte. No functional change except
pmap.h now includes pte.h.

Remove unused macros, and make sure the remaining one do not truncate the
high 32 bits of addresses.

More pmap changes from the potpourri pool:
- use wm_page pg_flags pmap-reserved flags to store attributes, instead of
defining a vm_page_md field.
- use atomic operations to touch the above mentioned flags.
- never create ptes with PG_RO and PG_M set (this was harmless anyway).
- in pmap_clear_modify(), do not flush cache if the page was mapped uncached.

Tested on r5k, rm5200, r10k and r12k.

Remove advertising clause from UCB licenses; ok deraad@

Fix some 64 bit address problems.
Some function names made more unique.
Other changes for the upcoming Origin 200 support.

Move <machine/pte.h> to mips64 land.