Ajust pmap_enter_ma() for upcoming new Xen privcmd ioctl:
pass flags to xpq_update_foreign()
Introduce a pmap MD flag: PMAP_MD_XEN_NOTR, which cause xpq_update_foreign()
to use the MMU_PT_UPDATE_NO_TRANSLATE flag.
make xpq_update_foreign() return the raw Xen error. This will cause
pmap_enter_ma() to return a negative error number in this case, but the
only user of this code path is privcmd.c and it can deal with it.

Add pmap_enter_gnt()m which maps a set of Xen grant entries at the
specified va in the specified pmap. Use the hooks implemented for EPT to
keep track of mapped grand entries in the pmap, and unmap them
when pmap_remove() is called. This requires pmap_remove() to be split
into a pmap_remove_locked(), to be called from pmap_remove_gnt().

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

branches: 1.42.6;
Switch to new PTE bits.

Further restrict the scope of XENPV to relevant parts.

Remove the non-PAE-i386 code of Xen. The branches are reordered so that
__x86_64__ comes first, eg:

#if defined(PAE)
/* i386+PAE */
#elif defined(__x86_64__)
/* amd64 */
#else
/* i386 */
#endif

becomes

#ifdef __x86_64__
/* amd64 */
#else
/* i386+PAE */
#endif

Tested on i386pae-domU and amd64-dom0.

branches: 1.39.12; 1.39.14;
A few changes:
* Use markers to reduce false sharing.
* Remove XENDEBUG_SYNC and several debug messages, they are just useless.
* Remove xen_vcpu_*. They are unused and not optimized: if we really
wanted to flush ranges we should pack the VAs in a mmuext_op array
instead of performing several hypercalls in a loop.
* Start removing PG_k.
* KNF, reorder, simplify and remove stupid comments.

branches: 1.38.4; 1.38.8; 1.38.12;
Use the hypervisor to copy/zero pages. This saves us the extra overheads
of setting up temporary kernel mapping/unmapping.

riz@ reports savings of about 2s on a 120s kernel build.

branches: 1.37.2; 1.37.4; 1.37.12;
Extend the xpmap API, as described in [1]. This change is mechanical and
avoids exposing the MD phys_to_machine/machine_to_phys tables directly.
Added:

- xpmap_ptom handles PFN (pseudo physical) to MFN (machine frame number)
translations, and is under control of the domain.
- xpmap_mtop is its counterpart (MFN to PFN), and is under control of
hypervisor.

xpmap_ptom_map() map a pseudo-phys address to a machine address
xpmap_ptom_unmap() unmap a pseudo-phys address (invalidation)
xpmap_ptom_isvalid() check for pseudo-phys address validity

The parameters are physical/machine addresses, like bus_dma/bus_space(9).
As x86 MFNs are tracked by u_long (Xen's choice) while machine addresses
can be 64 bits entities (PAE), use ptoa() to avoid truncation when bit
shifting by PAGE_SHIFT.

I kept the same namespace (xpmap_) to avoid code churn.

[1] http://mail-index.netbsd.org/port-xen/2009/05/09/msg004951.html

XXX will document ptoa/atop/trunc_page separately.

Retire XEN_COMPAT_030001 as detailed on port-xen@:

http://mail-index.netbsd.org/port-xen/2012/06/25/msg007431.html

The xen_p2m API comes next.

ok bouyer@.
Tested on i386 PAE and amd64 (Xen 3.3 on private test bed, and
Xen 3.4 for Amazon EC2).

FWIW, Amazon always reported:

hypervisor0 at mainbus0: Xen version 3.4.3-kaos_t1micro

multiple times for Europe and US West-1, so I guess they are now at
3.4 (32 and 64 bits).

Enable the map/unmap recursive mapping functions for all Xen ports for
save/restore.

For an unknown reason (to me) Xen refuses to update VM translations
when the entry is pointing back to itself (which is precisely
what our recursive VM model does). So enable the functions that take
care of this, which will avoid all sort of memory corruption upon restore
leading domU to trample upon itself.

Save/restore works again for amd64. The occasional domU frontend corruption is
still present, but is harmless to dom0. Now we have a working shell and
ddb inside domU, that helps debugging a tiny bit.

XXX pull-up to -6.

- Convert x86 MD code, mainly pmap(9) e.g. TLB shootdown code, to use
kcpuset(9) and thus replace hardcoded CPU bitmasks. This removes the
limitation of maximum CPUs.

- Support up to 256 CPUs on amd64 architecture by default.

Bug fixes, improvements, completion of Xen part and testing on 64-core
AMD Opteron(tm) Processor 6282 SE (also, as Xen HVM domU with 128 CPUs)
by Manuel Bouyer.

branches: 1.33.2;
per-cpu shadow directory pages should be updated locally via cross-calls. Do this.

branches: 1.32.2;
Move Xen-specific functions to Xen pmap. Requested by cherry@.

Un'ifdef XEN in xen_pmap.c, it is always defined there.

Expose the PG_k #define pt/pd bit to both xen and "baremetal" x86. This is required, since kernel pages are mapped with user permissions in XEN/amd64 since the VM kernel runs in ring3. Since XEN/i386(including PAE) runs in ring1, supervisor mode is appropriate for these ports. We need to share this since the pmap implementation is still shared. Once the xen implementation is sufficiently independant of the x86 one, this can be made private to xen/include/xenpmap.h

[merging from cherry-xenmp] Make the xen MMU op queue locking api private. Implement per-cpu queues.

branches: 1.29.2;
remove unnecessary locking overhead for UP

Introduce locking primitives for Xen pte operations, and xen helper calls for MP related MMU ops

branches: 1.27.2;
Apply DRY: xpmap_{mtop,ptom}() can reuse xpmap_{mtop,ptom}_masked() for
the frame number lookup.

No functional change.

apply some _KERNEL_OPT.

Use only one function to pin pages with Xen, and provide macros to
call it for different levels (L1 => L4).

Replace all calls to xpq_queue_pin_table(...) in MD code with these new
functions, with proper #ifdef'ing depending on $MACHINE.

Rationale:
- only one function to modify for logging
- pushes responsibility to caller for chosing the proper pin level, rather
than Xen internal functions; this makes the pin level explicit rather than
implicit.

Boot tested for dom0 i386/amd64, PAE included. No functional change intended.

branches: 1.24.4; 1.24.6; 1.24.8;
Remove 3rd and 4th clauses. OK cl@ (copyright holder).

remove Xen2 support.
ok bouyer@

More i386PAE fixes:
- x86_round_page, x86_trunc_page, x86_btop and x86_ptob macros are used with
physical addresses; cast to paddr_t instead of u_long. Issue pointed out
by jym@
- machine_to_phys_mapping[] is a long. This is fine as it holds page
frame numbers (and this fits in a 32bit int as physical addresses are
only 36bits), but cast to paddr_t before << PAGE_SHIFT
- xen_start_info.store_mfn is a long; cast it to paddr_t before << PAGE_SHIFT.
should fix issue pointed out by cegger@

branches: 1.21.2; 1.21.4; 1.21.8; 1.21.12;
- add mfn_to_pfn() and pfn_to_mfn() macros, for rapid conversion between
pseudo-physical and machine frame numbers.

- add HYPERVISOR_crash() for i386 and amd64. Intended to be used by a domain
to notify Xen that it crashed on purpose, and request a dump (if applicable).

No functional changes intended.

Reviewed by Christoph (cegger@).

- printf -> aprint_*
- fix and add comments
- make some panic/error messages more relevant
- remove last '\n' in DPRINTK() macros, not required as it is already part of format string.

No functional changes.

branches: 1.19.6; 1.19.10; 1.19.16;
Merge the bouyer-xeni386 branch. This brings in PAE support to NetBSD xeni386
(domU only). PAE support is enabled by 'options PAE', see the new XEN3PAE_DOMU
and INSTALL_XEN3PAE_DOMU kernel config files.

See the comments in arch/i386/include/{pte.h,pmap.h} to see how it works.
In short, we still handle it as a 2-level MMU, with the second level page
directory being 4 pages in size. pmap switching is done by switching the
L2 pages in the L3 entries, instead of loading %cr3. This is almost required
by Xen, which handle the last L2 page (the one mapping 0xc0000000 - 0xffffffff)
in a very special way. But this approach should also work for native PAE
support if ever supported (in fact, the pmap should almost suport native
PAE, what's missing is bootstrap code in locore.S).

Merge the bouyer-xeni386 branch to head, at tag bouyer-xeni386-merge1 (the
branch is still active and will see i386PAE support developement).
Sumary of changes:
- switch xeni386 to the x86/x86/pmap.c, and the xen/x86/x86_xpmap.c
pmap bootstrap.
- merge back most of xen/i386/ to i386/i386
- change the build to reduce diffs between i386 and amd64 in file locations
- remove include files that were identical to the i386/amd64 counterparts,
the build will find them via the xen-ma/machine link.

branches: 1.17.6;
Use the new atomic ops.

Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.

branches: 1.15.8; 1.15.14; 1.15.26; 1.15.28; 1.15.32; 1.15.34;
Add ELF_PADDR_OFFSET and VIRT_ENTRY strings to __xen_guest ELF section,
so that our kernels works with newer xen-3 hypervisors; and correct the value
of VIRT_BASE for dom0.
Now that we can embed the values of KERNBASE and KERNTEXTOFF in the binary
for Xen, make the domU memory layout the same as dom0 for Xen3 (making
it the other way round doens't work; probably because of alignement
constraints in the hypervisor). The old domU layout is used if options
XEN_COMPAT_030001 is present in the kernel config file. Enable this the
domU kernel config files for now, in case someone wants to run a NetBSD
domU on an older Xen3 installation.

branches: 1.14.12; 1.14.14;
Implement MULTI_update_va_mapping() and MULTI_update_va_mapping_otherdomain(),
which fills in multicall arguments for __HYPERVISOR_update_va_mapping
and __HYPERVISOR_update_va_mapping_otherdomain dealing with
differences between i386 and amd64.

branches: 1.13.2;
Change "inline" back to "__inline" in .h files -- C99 is still too
new, and some apps compile things in C89 mode. C89 keywords stay.

As per core@.

branches: 1.12.2; 1.12.4;
Snapshot of work in progress on NetBSD port to Xen3:
- kernel (both dom0 and domU) boot, console is functionnal and it can starts
software from a ramdisk
- there is no driver front-end expect console for domU yet.
- dom0 can probe devices and ex(4) work when Xen3 is booted without acpi
and apic support. But the on-board IDE doens't get interrupts.
The PCI code still needs work (it's hardcoded to mode 1). Some of this
code should be shared with ../x86
The physical insterrupt code needs to get MPBIOS and ACPI support, and
do interrupt routing to properly interract with Xen.
To enable Xen-3.0 support, add
options XEN3
to your kernel config file (this will disable Xen2 support)
Changes affecting Xen-2.0 support (no functionnal changes intended):
- get more constants from genassym for assembly code
- remove some unneeded registers move from start()
- map the shared info page from start(), and remove the pte = 0xffffffff hack
- vector.S: in hypervisor_callback() make sure %esi points to
HYPERVISOR_shared_info before accessing the info page. Remplace some
hand-written assembly with the equivalent macro defined in frameasm.h
- more debug code, dissabled by default.

while here added my copyright on some files I worked on in 2005.

branches: 1.11.2;
Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.

merge ktrace-lwp.

some assym cleanup.
- move copyin and friends from locore.S to their own file, copy.S.
share it between i386 and xen.
- defparam KERNBASE and kill KERNBASE_LOCORE hack.
- add more symbols to assym.h and use it where appropriate.

Implement xpq_queue_machphys_update(), which queues a request to
update the machine to physical table (to be used after a
MEMOP_increase_reservation).

branches: 1.7.2;
avoid variable shadowing.

Always call the xpq_queue*() functions at splvm(), so that it's safe to call
them from interrupt context.
xpq_flush_queue() is called from IPL_NET in if_xennet.c, and
other xpq_queue* functions may be called from interrupt context via
pmap_kenter*(). Should fix port-xen/30153.
Thanks to Jason Thorpe and YAMAMOTO Takashi for enlightments on this issue.

s/foreing/foreign/g
ok'ed by Manuel Bouyer.

branches: 1.4.2;
Merge the bouyer-xen2 branch. This add supports for the Xen 2.0 virtual
machine kernel (both privileged and non-privileged domains), and remove support
for the old xen 1.2.

branches: 1.3.2; 1.3.6; 1.3.8; 1.3.10;
Rework the physical<->machine memory mapping: offset physical addresses
by 0x100000 (above the I/O Memory "hole") leaving all physical addresses
below unused, don't perform phys<->mach mapping for addresses below 0x100000
or beyond the real hardware's physical memory.

-> /dev/mem works now as expected and X works in domain0.

Make bus_space map machine addresses instead of physical addresses.

branches: 1.1.2;
Add port to the Xen virtual machine monitor.
(see http://www.cl.cam.ac.uk/Research/SRG/netos/xen/)

Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor

branches: 1.42.6;
Switch to new PTE bits.

Further restrict the scope of XENPV to relevant parts.

Remove the non-PAE-i386 code of Xen. The branches are reordered so that
__x86_64__ comes first, eg:

#if defined(PAE)
/* i386+PAE */
#elif defined(__x86_64__)
/* amd64 */
#else
/* i386 */
#endif

becomes

#ifdef __x86_64__
/* amd64 */
#else
/* i386+PAE */
#endif

Tested on i386pae-domU and amd64-dom0.

branches: 1.39.12; 1.39.14;
A few changes:
* Use markers to reduce false sharing.
* Remove XENDEBUG_SYNC and several debug messages, they are just useless.
* Remove xen_vcpu_*. They are unused and not optimized: if we really
wanted to flush ranges we should pack the VAs in a mmuext_op array
instead of performing several hypercalls in a loop.
* Start removing PG_k.
* KNF, reorder, simplify and remove stupid comments.

branches: 1.38.4; 1.38.8; 1.38.12;
Use the hypervisor to copy/zero pages. This saves us the extra overheads
of setting up temporary kernel mapping/unmapping.

riz@ reports savings of about 2s on a 120s kernel build.

branches: 1.37.2; 1.37.4; 1.37.12;
Extend the xpmap API, as described in [1]. This change is mechanical and
avoids exposing the MD phys_to_machine/machine_to_phys tables directly.
Added:

- xpmap_ptom handles PFN (pseudo physical) to MFN (machine frame number)
translations, and is under control of the domain.
- xpmap_mtop is its counterpart (MFN to PFN), and is under control of
hypervisor.

xpmap_ptom_map() map a pseudo-phys address to a machine address
xpmap_ptom_unmap() unmap a pseudo-phys address (invalidation)
xpmap_ptom_isvalid() check for pseudo-phys address validity

The parameters are physical/machine addresses, like bus_dma/bus_space(9).
As x86 MFNs are tracked by u_long (Xen's choice) while machine addresses
can be 64 bits entities (PAE), use ptoa() to avoid truncation when bit
shifting by PAGE_SHIFT.

I kept the same namespace (xpmap_) to avoid code churn.

[1] http://mail-index.netbsd.org/port-xen/2009/05/09/msg004951.html

XXX will document ptoa/atop/trunc_page separately.

Retire XEN_COMPAT_030001 as detailed on port-xen@:

http://mail-index.netbsd.org/port-xen/2012/06/25/msg007431.html

The xen_p2m API comes next.

ok bouyer@.
Tested on i386 PAE and amd64 (Xen 3.3 on private test bed, and
Xen 3.4 for Amazon EC2).

FWIW, Amazon always reported:

hypervisor0 at mainbus0: Xen version 3.4.3-kaos_t1micro

multiple times for Europe and US West-1, so I guess they are now at
3.4 (32 and 64 bits).

Enable the map/unmap recursive mapping functions for all Xen ports for
save/restore.

For an unknown reason (to me) Xen refuses to update VM translations
when the entry is pointing back to itself (which is precisely
what our recursive VM model does). So enable the functions that take
care of this, which will avoid all sort of memory corruption upon restore
leading domU to trample upon itself.

Save/restore works again for amd64. The occasional domU frontend corruption is
still present, but is harmless to dom0. Now we have a working shell and
ddb inside domU, that helps debugging a tiny bit.

XXX pull-up to -6.

- Convert x86 MD code, mainly pmap(9) e.g. TLB shootdown code, to use
kcpuset(9) and thus replace hardcoded CPU bitmasks. This removes the
limitation of maximum CPUs.

- Support up to 256 CPUs on amd64 architecture by default.

Bug fixes, improvements, completion of Xen part and testing on 64-core
AMD Opteron(tm) Processor 6282 SE (also, as Xen HVM domU with 128 CPUs)
by Manuel Bouyer.

branches: 1.33.2;
per-cpu shadow directory pages should be updated locally via cross-calls. Do this.

branches: 1.32.2;
Move Xen-specific functions to Xen pmap. Requested by cherry@.

Un'ifdef XEN in xen_pmap.c, it is always defined there.

Expose the PG_k #define pt/pd bit to both xen and "baremetal" x86. This is required, since kernel pages are mapped with user permissions in XEN/amd64 since the VM kernel runs in ring3. Since XEN/i386(including PAE) runs in ring1, supervisor mode is appropriate for these ports. We need to share this since the pmap implementation is still shared. Once the xen implementation is sufficiently independant of the x86 one, this can be made private to xen/include/xenpmap.h

[merging from cherry-xenmp] Make the xen MMU op queue locking api private. Implement per-cpu queues.

branches: 1.29.2;
remove unnecessary locking overhead for UP

Introduce locking primitives for Xen pte operations, and xen helper calls for MP related MMU ops

branches: 1.27.2;
Apply DRY: xpmap_{mtop,ptom}() can reuse xpmap_{mtop,ptom}_masked() for
the frame number lookup.

No functional change.

apply some _KERNEL_OPT.

Use only one function to pin pages with Xen, and provide macros to
call it for different levels (L1 => L4).

Replace all calls to xpq_queue_pin_table(...) in MD code with these new
functions, with proper #ifdef'ing depending on $MACHINE.

Rationale:
- only one function to modify for logging
- pushes responsibility to caller for chosing the proper pin level, rather
than Xen internal functions; this makes the pin level explicit rather than
implicit.

Boot tested for dom0 i386/amd64, PAE included. No functional change intended.

branches: 1.24.4; 1.24.6; 1.24.8;
Remove 3rd and 4th clauses. OK cl@ (copyright holder).

remove Xen2 support.
ok bouyer@

More i386PAE fixes:
- x86_round_page, x86_trunc_page, x86_btop and x86_ptob macros are used with
physical addresses; cast to paddr_t instead of u_long. Issue pointed out
by jym@
- machine_to_phys_mapping[] is a long. This is fine as it holds page
frame numbers (and this fits in a 32bit int as physical addresses are
only 36bits), but cast to paddr_t before << PAGE_SHIFT
- xen_start_info.store_mfn is a long; cast it to paddr_t before << PAGE_SHIFT.
should fix issue pointed out by cegger@

branches: 1.21.2; 1.21.4; 1.21.8; 1.21.12;
- add mfn_to_pfn() and pfn_to_mfn() macros, for rapid conversion between
pseudo-physical and machine frame numbers.

- add HYPERVISOR_crash() for i386 and amd64. Intended to be used by a domain
to notify Xen that it crashed on purpose, and request a dump (if applicable).

No functional changes intended.

Reviewed by Christoph (cegger@).

- printf -> aprint_*
- fix and add comments
- make some panic/error messages more relevant
- remove last '\n' in DPRINTK() macros, not required as it is already part of format string.

No functional changes.

branches: 1.19.6; 1.19.10; 1.19.16;
Merge the bouyer-xeni386 branch. This brings in PAE support to NetBSD xeni386
(domU only). PAE support is enabled by 'options PAE', see the new XEN3PAE_DOMU
and INSTALL_XEN3PAE_DOMU kernel config files.

See the comments in arch/i386/include/{pte.h,pmap.h} to see how it works.
In short, we still handle it as a 2-level MMU, with the second level page
directory being 4 pages in size. pmap switching is done by switching the
L2 pages in the L3 entries, instead of loading %cr3. This is almost required
by Xen, which handle the last L2 page (the one mapping 0xc0000000 - 0xffffffff)
in a very special way. But this approach should also work for native PAE
support if ever supported (in fact, the pmap should almost suport native
PAE, what's missing is bootstrap code in locore.S).

Merge the bouyer-xeni386 branch to head, at tag bouyer-xeni386-merge1 (the
branch is still active and will see i386PAE support developement).
Sumary of changes:
- switch xeni386 to the x86/x86/pmap.c, and the xen/x86/x86_xpmap.c
pmap bootstrap.
- merge back most of xen/i386/ to i386/i386
- change the build to reduce diffs between i386 and amd64 in file locations
- remove include files that were identical to the i386/amd64 counterparts,
the build will find them via the xen-ma/machine link.

branches: 1.17.6;
Use the new atomic ops.

Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.

branches: 1.15.8; 1.15.14; 1.15.26; 1.15.28; 1.15.32; 1.15.34;
Add ELF_PADDR_OFFSET and VIRT_ENTRY strings to __xen_guest ELF section,
so that our kernels works with newer xen-3 hypervisors; and correct the value
of VIRT_BASE for dom0.
Now that we can embed the values of KERNBASE and KERNTEXTOFF in the binary
for Xen, make the domU memory layout the same as dom0 for Xen3 (making
it the other way round doens't work; probably because of alignement
constraints in the hypervisor). The old domU layout is used if options
XEN_COMPAT_030001 is present in the kernel config file. Enable this the
domU kernel config files for now, in case someone wants to run a NetBSD
domU on an older Xen3 installation.

branches: 1.14.12; 1.14.14;
Implement MULTI_update_va_mapping() and MULTI_update_va_mapping_otherdomain(),
which fills in multicall arguments for __HYPERVISOR_update_va_mapping
and __HYPERVISOR_update_va_mapping_otherdomain dealing with
differences between i386 and amd64.

branches: 1.13.2;
Change "inline" back to "__inline" in .h files -- C99 is still too
new, and some apps compile things in C89 mode. C89 keywords stay.

As per core@.

branches: 1.12.2; 1.12.4;
Snapshot of work in progress on NetBSD port to Xen3:
- kernel (both dom0 and domU) boot, console is functionnal and it can starts
software from a ramdisk
- there is no driver front-end expect console for domU yet.
- dom0 can probe devices and ex(4) work when Xen3 is booted without acpi
and apic support. But the on-board IDE doens't get interrupts.
The PCI code still needs work (it's hardcoded to mode 1). Some of this
code should be shared with ../x86
The physical insterrupt code needs to get MPBIOS and ACPI support, and
do interrupt routing to properly interract with Xen.
To enable Xen-3.0 support, add
options XEN3
to your kernel config file (this will disable Xen2 support)
Changes affecting Xen-2.0 support (no functionnal changes intended):
- get more constants from genassym for assembly code
- remove some unneeded registers move from start()
- map the shared info page from start(), and remove the pte = 0xffffffff hack
- vector.S: in hypervisor_callback() make sure %esi points to
HYPERVISOR_shared_info before accessing the info page. Remplace some
hand-written assembly with the equivalent macro defined in frameasm.h
- more debug code, dissabled by default.

while here added my copyright on some files I worked on in 2005.

branches: 1.11.2;
Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.

merge ktrace-lwp.

some assym cleanup.
- move copyin and friends from locore.S to their own file, copy.S.
share it between i386 and xen.
- defparam KERNBASE and kill KERNBASE_LOCORE hack.
- add more symbols to assym.h and use it where appropriate.

Implement xpq_queue_machphys_update(), which queues a request to
update the machine to physical table (to be used after a
MEMOP_increase_reservation).

branches: 1.7.2;
avoid variable shadowing.

Always call the xpq_queue*() functions at splvm(), so that it's safe to call
them from interrupt context.
xpq_flush_queue() is called from IPL_NET in if_xennet.c, and
other xpq_queue* functions may be called from interrupt context via
pmap_kenter*(). Should fix port-xen/30153.
Thanks to Jason Thorpe and YAMAMOTO Takashi for enlightments on this issue.

s/foreing/foreign/g
ok'ed by Manuel Bouyer.

branches: 1.4.2;
Merge the bouyer-xen2 branch. This add supports for the Xen 2.0 virtual
machine kernel (both privileged and non-privileged domains), and remove support
for the old xen 1.2.

branches: 1.3.2; 1.3.6; 1.3.8; 1.3.10;
Rework the physical<->machine memory mapping: offset physical addresses
by 0x100000 (above the I/O Memory "hole") leaving all physical addresses
below unused, don't perform phys<->mach mapping for addresses below 0x100000
or beyond the real hardware's physical memory.

-> /dev/mem works now as expected and X works in domain0.

Make bus_space map machine addresses instead of physical addresses.

branches: 1.1.2;
Add port to the Xen virtual machine monitor.
(see http://www.cl.cam.ac.uk/Research/SRG/netos/xen/)

Switch to new PTE bits.

Further restrict the scope of XENPV to relevant parts.

Remove the non-PAE-i386 code of Xen. The branches are reordered so that
__x86_64__ comes first, eg:

#if defined(PAE)
/* i386+PAE */
#elif defined(__x86_64__)
/* amd64 */
#else
/* i386 */
#endif

becomes

#ifdef __x86_64__
/* amd64 */
#else
/* i386+PAE */
#endif

Tested on i386pae-domU and amd64-dom0.

branches: 1.39.12; 1.39.14;
A few changes:
* Use markers to reduce false sharing.
* Remove XENDEBUG_SYNC and several debug messages, they are just useless.
* Remove xen_vcpu_*. They are unused and not optimized: if we really
wanted to flush ranges we should pack the VAs in a mmuext_op array
instead of performing several hypercalls in a loop.
* Start removing PG_k.
* KNF, reorder, simplify and remove stupid comments.

branches: 1.38.4; 1.38.8; 1.38.12;
Use the hypervisor to copy/zero pages. This saves us the extra overheads
of setting up temporary kernel mapping/unmapping.

riz@ reports savings of about 2s on a 120s kernel build.

branches: 1.37.2; 1.37.4; 1.37.12;
Extend the xpmap API, as described in [1]. This change is mechanical and
avoids exposing the MD phys_to_machine/machine_to_phys tables directly.
Added:

- xpmap_ptom handles PFN (pseudo physical) to MFN (machine frame number)
translations, and is under control of the domain.
- xpmap_mtop is its counterpart (MFN to PFN), and is under control of
hypervisor.

xpmap_ptom_map() map a pseudo-phys address to a machine address
xpmap_ptom_unmap() unmap a pseudo-phys address (invalidation)
xpmap_ptom_isvalid() check for pseudo-phys address validity

The parameters are physical/machine addresses, like bus_dma/bus_space(9).
As x86 MFNs are tracked by u_long (Xen's choice) while machine addresses
can be 64 bits entities (PAE), use ptoa() to avoid truncation when bit
shifting by PAGE_SHIFT.

I kept the same namespace (xpmap_) to avoid code churn.

[1] http://mail-index.netbsd.org/port-xen/2009/05/09/msg004951.html

XXX will document ptoa/atop/trunc_page separately.

Retire XEN_COMPAT_030001 as detailed on port-xen@:

http://mail-index.netbsd.org/port-xen/2012/06/25/msg007431.html

The xen_p2m API comes next.

ok bouyer@.
Tested on i386 PAE and amd64 (Xen 3.3 on private test bed, and
Xen 3.4 for Amazon EC2).

FWIW, Amazon always reported:

hypervisor0 at mainbus0: Xen version 3.4.3-kaos_t1micro

multiple times for Europe and US West-1, so I guess they are now at
3.4 (32 and 64 bits).

Enable the map/unmap recursive mapping functions for all Xen ports for
save/restore.

For an unknown reason (to me) Xen refuses to update VM translations
when the entry is pointing back to itself (which is precisely
what our recursive VM model does). So enable the functions that take
care of this, which will avoid all sort of memory corruption upon restore
leading domU to trample upon itself.

Save/restore works again for amd64. The occasional domU frontend corruption is
still present, but is harmless to dom0. Now we have a working shell and
ddb inside domU, that helps debugging a tiny bit.

XXX pull-up to -6.

- Convert x86 MD code, mainly pmap(9) e.g. TLB shootdown code, to use
kcpuset(9) and thus replace hardcoded CPU bitmasks. This removes the
limitation of maximum CPUs.

- Support up to 256 CPUs on amd64 architecture by default.

Bug fixes, improvements, completion of Xen part and testing on 64-core
AMD Opteron(tm) Processor 6282 SE (also, as Xen HVM domU with 128 CPUs)
by Manuel Bouyer.

branches: 1.33.2;
per-cpu shadow directory pages should be updated locally via cross-calls. Do this.

branches: 1.32.2;
Move Xen-specific functions to Xen pmap. Requested by cherry@.

Un'ifdef XEN in xen_pmap.c, it is always defined there.

Expose the PG_k #define pt/pd bit to both xen and "baremetal" x86. This is required, since kernel pages are mapped with user permissions in XEN/amd64 since the VM kernel runs in ring3. Since XEN/i386(including PAE) runs in ring1, supervisor mode is appropriate for these ports. We need to share this since the pmap implementation is still shared. Once the xen implementation is sufficiently independant of the x86 one, this can be made private to xen/include/xenpmap.h

[merging from cherry-xenmp] Make the xen MMU op queue locking api private. Implement per-cpu queues.

branches: 1.29.2;
remove unnecessary locking overhead for UP

Introduce locking primitives for Xen pte operations, and xen helper calls for MP related MMU ops

branches: 1.27.2;
Apply DRY: xpmap_{mtop,ptom}() can reuse xpmap_{mtop,ptom}_masked() for
the frame number lookup.

No functional change.

apply some _KERNEL_OPT.

Use only one function to pin pages with Xen, and provide macros to
call it for different levels (L1 => L4).

Replace all calls to xpq_queue_pin_table(...) in MD code with these new
functions, with proper #ifdef'ing depending on $MACHINE.

Rationale:
- only one function to modify for logging
- pushes responsibility to caller for chosing the proper pin level, rather
than Xen internal functions; this makes the pin level explicit rather than
implicit.

Boot tested for dom0 i386/amd64, PAE included. No functional change intended.

branches: 1.24.4; 1.24.6; 1.24.8;
Remove 3rd and 4th clauses. OK cl@ (copyright holder).

remove Xen2 support.
ok bouyer@

More i386PAE fixes:
- x86_round_page, x86_trunc_page, x86_btop and x86_ptob macros are used with
physical addresses; cast to paddr_t instead of u_long. Issue pointed out
by jym@
- machine_to_phys_mapping[] is a long. This is fine as it holds page
frame numbers (and this fits in a 32bit int as physical addresses are
only 36bits), but cast to paddr_t before << PAGE_SHIFT
- xen_start_info.store_mfn is a long; cast it to paddr_t before << PAGE_SHIFT.
should fix issue pointed out by cegger@

branches: 1.21.2; 1.21.4; 1.21.8; 1.21.12;
- add mfn_to_pfn() and pfn_to_mfn() macros, for rapid conversion between
pseudo-physical and machine frame numbers.

- add HYPERVISOR_crash() for i386 and amd64. Intended to be used by a domain
to notify Xen that it crashed on purpose, and request a dump (if applicable).

No functional changes intended.

Reviewed by Christoph (cegger@).

- printf -> aprint_*
- fix and add comments
- make some panic/error messages more relevant
- remove last '\n' in DPRINTK() macros, not required as it is already part of format string.

No functional changes.

branches: 1.19.6; 1.19.10; 1.19.16;
Merge the bouyer-xeni386 branch. This brings in PAE support to NetBSD xeni386
(domU only). PAE support is enabled by 'options PAE', see the new XEN3PAE_DOMU
and INSTALL_XEN3PAE_DOMU kernel config files.

See the comments in arch/i386/include/{pte.h,pmap.h} to see how it works.
In short, we still handle it as a 2-level MMU, with the second level page
directory being 4 pages in size. pmap switching is done by switching the
L2 pages in the L3 entries, instead of loading %cr3. This is almost required
by Xen, which handle the last L2 page (the one mapping 0xc0000000 - 0xffffffff)
in a very special way. But this approach should also work for native PAE
support if ever supported (in fact, the pmap should almost suport native
PAE, what's missing is bootstrap code in locore.S).

Merge the bouyer-xeni386 branch to head, at tag bouyer-xeni386-merge1 (the
branch is still active and will see i386PAE support developement).
Sumary of changes:
- switch xeni386 to the x86/x86/pmap.c, and the xen/x86/x86_xpmap.c
pmap bootstrap.
- merge back most of xen/i386/ to i386/i386
- change the build to reduce diffs between i386 and amd64 in file locations
- remove include files that were identical to the i386/amd64 counterparts,
the build will find them via the xen-ma/machine link.

branches: 1.17.6;
Use the new atomic ops.

Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.

branches: 1.15.8; 1.15.14; 1.15.26; 1.15.28; 1.15.32; 1.15.34;
Add ELF_PADDR_OFFSET and VIRT_ENTRY strings to __xen_guest ELF section,
so that our kernels works with newer xen-3 hypervisors; and correct the value
of VIRT_BASE for dom0.
Now that we can embed the values of KERNBASE and KERNTEXTOFF in the binary
for Xen, make the domU memory layout the same as dom0 for Xen3 (making
it the other way round doens't work; probably because of alignement
constraints in the hypervisor). The old domU layout is used if options
XEN_COMPAT_030001 is present in the kernel config file. Enable this the
domU kernel config files for now, in case someone wants to run a NetBSD
domU on an older Xen3 installation.

branches: 1.14.12; 1.14.14;
Implement MULTI_update_va_mapping() and MULTI_update_va_mapping_otherdomain(),
which fills in multicall arguments for __HYPERVISOR_update_va_mapping
and __HYPERVISOR_update_va_mapping_otherdomain dealing with
differences between i386 and amd64.

branches: 1.13.2;
Change "inline" back to "__inline" in .h files -- C99 is still too
new, and some apps compile things in C89 mode. C89 keywords stay.

As per core@.

branches: 1.12.2; 1.12.4;
Snapshot of work in progress on NetBSD port to Xen3:
- kernel (both dom0 and domU) boot, console is functionnal and it can starts
software from a ramdisk
- there is no driver front-end expect console for domU yet.
- dom0 can probe devices and ex(4) work when Xen3 is booted without acpi
and apic support. But the on-board IDE doens't get interrupts.
The PCI code still needs work (it's hardcoded to mode 1). Some of this
code should be shared with ../x86
The physical insterrupt code needs to get MPBIOS and ACPI support, and
do interrupt routing to properly interract with Xen.
To enable Xen-3.0 support, add
options XEN3
to your kernel config file (this will disable Xen2 support)
Changes affecting Xen-2.0 support (no functionnal changes intended):
- get more constants from genassym for assembly code
- remove some unneeded registers move from start()
- map the shared info page from start(), and remove the pte = 0xffffffff hack
- vector.S: in hypervisor_callback() make sure %esi points to
HYPERVISOR_shared_info before accessing the info page. Remplace some
hand-written assembly with the equivalent macro defined in frameasm.h
- more debug code, dissabled by default.

while here added my copyright on some files I worked on in 2005.

branches: 1.11.2;
Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.

merge ktrace-lwp.

some assym cleanup.
- move copyin and friends from locore.S to their own file, copy.S.
share it between i386 and xen.
- defparam KERNBASE and kill KERNBASE_LOCORE hack.
- add more symbols to assym.h and use it where appropriate.

Implement xpq_queue_machphys_update(), which queues a request to
update the machine to physical table (to be used after a
MEMOP_increase_reservation).

branches: 1.7.2;
avoid variable shadowing.

Always call the xpq_queue*() functions at splvm(), so that it's safe to call
them from interrupt context.
xpq_flush_queue() is called from IPL_NET in if_xennet.c, and
other xpq_queue* functions may be called from interrupt context via
pmap_kenter*(). Should fix port-xen/30153.
Thanks to Jason Thorpe and YAMAMOTO Takashi for enlightments on this issue.

s/foreing/foreign/g
ok'ed by Manuel Bouyer.

branches: 1.4.2;
Merge the bouyer-xen2 branch. This add supports for the Xen 2.0 virtual
machine kernel (both privileged and non-privileged domains), and remove support
for the old xen 1.2.

branches: 1.3.2; 1.3.6; 1.3.8; 1.3.10;
Rework the physical<->machine memory mapping: offset physical addresses
by 0x100000 (above the I/O Memory "hole") leaving all physical addresses
below unused, don't perform phys<->mach mapping for addresses below 0x100000
or beyond the real hardware's physical memory.

-> /dev/mem works now as expected and X works in domain0.

Make bus_space map machine addresses instead of physical addresses.

branches: 1.1.2;
Add port to the Xen virtual machine monitor.
(see http://www.cl.cam.ac.uk/Research/SRG/netos/xen/)

Further restrict the scope of XENPV to relevant parts.

Remove the non-PAE-i386 code of Xen. The branches are reordered so that
__x86_64__ comes first, eg:

#if defined(PAE)
/* i386+PAE */
#elif defined(__x86_64__)
/* amd64 */
#else
/* i386 */
#endif

becomes

#ifdef __x86_64__
/* amd64 */
#else
/* i386+PAE */
#endif

Tested on i386pae-domU and amd64-dom0.

branches: 1.39.12;
A few changes:
* Use markers to reduce false sharing.
* Remove XENDEBUG_SYNC and several debug messages, they are just useless.
* Remove xen_vcpu_*. They are unused and not optimized: if we really
wanted to flush ranges we should pack the VAs in a mmuext_op array
instead of performing several hypercalls in a loop.
* Start removing PG_k.
* KNF, reorder, simplify and remove stupid comments.

branches: 1.38.4; 1.38.8; 1.38.12;
Use the hypervisor to copy/zero pages. This saves us the extra overheads
of setting up temporary kernel mapping/unmapping.

riz@ reports savings of about 2s on a 120s kernel build.

branches: 1.37.2; 1.37.4; 1.37.12;
Extend the xpmap API, as described in [1]. This change is mechanical and
avoids exposing the MD phys_to_machine/machine_to_phys tables directly.
Added:

- xpmap_ptom handles PFN (pseudo physical) to MFN (machine frame number)
translations, and is under control of the domain.
- xpmap_mtop is its counterpart (MFN to PFN), and is under control of
hypervisor.

xpmap_ptom_map() map a pseudo-phys address to a machine address
xpmap_ptom_unmap() unmap a pseudo-phys address (invalidation)
xpmap_ptom_isvalid() check for pseudo-phys address validity

The parameters are physical/machine addresses, like bus_dma/bus_space(9).
As x86 MFNs are tracked by u_long (Xen's choice) while machine addresses
can be 64 bits entities (PAE), use ptoa() to avoid truncation when bit
shifting by PAGE_SHIFT.

I kept the same namespace (xpmap_) to avoid code churn.

[1] http://mail-index.netbsd.org/port-xen/2009/05/09/msg004951.html

XXX will document ptoa/atop/trunc_page separately.

Retire XEN_COMPAT_030001 as detailed on port-xen@:

http://mail-index.netbsd.org/port-xen/2012/06/25/msg007431.html

The xen_p2m API comes next.

ok bouyer@.
Tested on i386 PAE and amd64 (Xen 3.3 on private test bed, and
Xen 3.4 for Amazon EC2).

FWIW, Amazon always reported:

hypervisor0 at mainbus0: Xen version 3.4.3-kaos_t1micro

multiple times for Europe and US West-1, so I guess they are now at
3.4 (32 and 64 bits).

Enable the map/unmap recursive mapping functions for all Xen ports for
save/restore.

For an unknown reason (to me) Xen refuses to update VM translations
when the entry is pointing back to itself (which is precisely
what our recursive VM model does). So enable the functions that take
care of this, which will avoid all sort of memory corruption upon restore
leading domU to trample upon itself.

Save/restore works again for amd64. The occasional domU frontend corruption is
still present, but is harmless to dom0. Now we have a working shell and
ddb inside domU, that helps debugging a tiny bit.

XXX pull-up to -6.

- Convert x86 MD code, mainly pmap(9) e.g. TLB shootdown code, to use
kcpuset(9) and thus replace hardcoded CPU bitmasks. This removes the
limitation of maximum CPUs.

- Support up to 256 CPUs on amd64 architecture by default.

Bug fixes, improvements, completion of Xen part and testing on 64-core
AMD Opteron(tm) Processor 6282 SE (also, as Xen HVM domU with 128 CPUs)
by Manuel Bouyer.

branches: 1.33.2;
per-cpu shadow directory pages should be updated locally via cross-calls. Do this.

branches: 1.32.2;
Move Xen-specific functions to Xen pmap. Requested by cherry@.

Un'ifdef XEN in xen_pmap.c, it is always defined there.

Expose the PG_k #define pt/pd bit to both xen and "baremetal" x86. This is required, since kernel pages are mapped with user permissions in XEN/amd64 since the VM kernel runs in ring3. Since XEN/i386(including PAE) runs in ring1, supervisor mode is appropriate for these ports. We need to share this since the pmap implementation is still shared. Once the xen implementation is sufficiently independant of the x86 one, this can be made private to xen/include/xenpmap.h

[merging from cherry-xenmp] Make the xen MMU op queue locking api private. Implement per-cpu queues.

branches: 1.29.2;
remove unnecessary locking overhead for UP

Introduce locking primitives for Xen pte operations, and xen helper calls for MP related MMU ops

branches: 1.27.2;
Apply DRY: xpmap_{mtop,ptom}() can reuse xpmap_{mtop,ptom}_masked() for
the frame number lookup.

No functional change.

apply some _KERNEL_OPT.

Use only one function to pin pages with Xen, and provide macros to
call it for different levels (L1 => L4).

Replace all calls to xpq_queue_pin_table(...) in MD code with these new
functions, with proper #ifdef'ing depending on $MACHINE.

Rationale:
- only one function to modify for logging
- pushes responsibility to caller for chosing the proper pin level, rather
than Xen internal functions; this makes the pin level explicit rather than
implicit.

Boot tested for dom0 i386/amd64, PAE included. No functional change intended.

branches: 1.24.4; 1.24.6; 1.24.8;
Remove 3rd and 4th clauses. OK cl@ (copyright holder).

remove Xen2 support.
ok bouyer@

More i386PAE fixes:
- x86_round_page, x86_trunc_page, x86_btop and x86_ptob macros are used with
physical addresses; cast to paddr_t instead of u_long. Issue pointed out
by jym@
- machine_to_phys_mapping[] is a long. This is fine as it holds page
frame numbers (and this fits in a 32bit int as physical addresses are
only 36bits), but cast to paddr_t before << PAGE_SHIFT
- xen_start_info.store_mfn is a long; cast it to paddr_t before << PAGE_SHIFT.
should fix issue pointed out by cegger@

branches: 1.21.2; 1.21.4; 1.21.8; 1.21.12;
- add mfn_to_pfn() and pfn_to_mfn() macros, for rapid conversion between
pseudo-physical and machine frame numbers.

- add HYPERVISOR_crash() for i386 and amd64. Intended to be used by a domain
to notify Xen that it crashed on purpose, and request a dump (if applicable).

No functional changes intended.

Reviewed by Christoph (cegger@).

- printf -> aprint_*
- fix and add comments
- make some panic/error messages more relevant
- remove last '\n' in DPRINTK() macros, not required as it is already part of format string.

No functional changes.

branches: 1.19.6; 1.19.10; 1.19.16;
Merge the bouyer-xeni386 branch. This brings in PAE support to NetBSD xeni386
(domU only). PAE support is enabled by 'options PAE', see the new XEN3PAE_DOMU
and INSTALL_XEN3PAE_DOMU kernel config files.

See the comments in arch/i386/include/{pte.h,pmap.h} to see how it works.
In short, we still handle it as a 2-level MMU, with the second level page
directory being 4 pages in size. pmap switching is done by switching the
L2 pages in the L3 entries, instead of loading %cr3. This is almost required
by Xen, which handle the last L2 page (the one mapping 0xc0000000 - 0xffffffff)
in a very special way. But this approach should also work for native PAE
support if ever supported (in fact, the pmap should almost suport native
PAE, what's missing is bootstrap code in locore.S).

Merge the bouyer-xeni386 branch to head, at tag bouyer-xeni386-merge1 (the
branch is still active and will see i386PAE support developement).
Sumary of changes:
- switch xeni386 to the x86/x86/pmap.c, and the xen/x86/x86_xpmap.c
pmap bootstrap.
- merge back most of xen/i386/ to i386/i386
- change the build to reduce diffs between i386 and amd64 in file locations
- remove include files that were identical to the i386/amd64 counterparts,
the build will find them via the xen-ma/machine link.

branches: 1.17.6;
Use the new atomic ops.

Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.

branches: 1.15.8; 1.15.14; 1.15.26; 1.15.28; 1.15.32; 1.15.34;
Add ELF_PADDR_OFFSET and VIRT_ENTRY strings to __xen_guest ELF section,
so that our kernels works with newer xen-3 hypervisors; and correct the value
of VIRT_BASE for dom0.
Now that we can embed the values of KERNBASE and KERNTEXTOFF in the binary
for Xen, make the domU memory layout the same as dom0 for Xen3 (making
it the other way round doens't work; probably because of alignement
constraints in the hypervisor). The old domU layout is used if options
XEN_COMPAT_030001 is present in the kernel config file. Enable this the
domU kernel config files for now, in case someone wants to run a NetBSD
domU on an older Xen3 installation.

branches: 1.14.12; 1.14.14;
Implement MULTI_update_va_mapping() and MULTI_update_va_mapping_otherdomain(),
which fills in multicall arguments for __HYPERVISOR_update_va_mapping
and __HYPERVISOR_update_va_mapping_otherdomain dealing with
differences between i386 and amd64.

branches: 1.13.2;
Change "inline" back to "__inline" in .h files -- C99 is still too
new, and some apps compile things in C89 mode. C89 keywords stay.

As per core@.

branches: 1.12.2; 1.12.4;
Snapshot of work in progress on NetBSD port to Xen3:
- kernel (both dom0 and domU) boot, console is functionnal and it can starts
software from a ramdisk
- there is no driver front-end expect console for domU yet.
- dom0 can probe devices and ex(4) work when Xen3 is booted without acpi
and apic support. But the on-board IDE doens't get interrupts.
The PCI code still needs work (it's hardcoded to mode 1). Some of this
code should be shared with ../x86
The physical insterrupt code needs to get MPBIOS and ACPI support, and
do interrupt routing to properly interract with Xen.
To enable Xen-3.0 support, add
options XEN3
to your kernel config file (this will disable Xen2 support)
Changes affecting Xen-2.0 support (no functionnal changes intended):
- get more constants from genassym for assembly code
- remove some unneeded registers move from start()
- map the shared info page from start(), and remove the pte = 0xffffffff hack
- vector.S: in hypervisor_callback() make sure %esi points to
HYPERVISOR_shared_info before accessing the info page. Remplace some
hand-written assembly with the equivalent macro defined in frameasm.h
- more debug code, dissabled by default.

while here added my copyright on some files I worked on in 2005.

branches: 1.11.2;
Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.

merge ktrace-lwp.

some assym cleanup.
- move copyin and friends from locore.S to their own file, copy.S.
share it between i386 and xen.
- defparam KERNBASE and kill KERNBASE_LOCORE hack.
- add more symbols to assym.h and use it where appropriate.

Implement xpq_queue_machphys_update(), which queues a request to
update the machine to physical table (to be used after a
MEMOP_increase_reservation).

branches: 1.7.2;
avoid variable shadowing.

Always call the xpq_queue*() functions at splvm(), so that it's safe to call
them from interrupt context.
xpq_flush_queue() is called from IPL_NET in if_xennet.c, and
other xpq_queue* functions may be called from interrupt context via
pmap_kenter*(). Should fix port-xen/30153.
Thanks to Jason Thorpe and YAMAMOTO Takashi for enlightments on this issue.

s/foreing/foreign/g
ok'ed by Manuel Bouyer.

branches: 1.4.2;
Merge the bouyer-xen2 branch. This add supports for the Xen 2.0 virtual
machine kernel (both privileged and non-privileged domains), and remove support
for the old xen 1.2.

branches: 1.3.2; 1.3.6; 1.3.8; 1.3.10;
Rework the physical<->machine memory mapping: offset physical addresses
by 0x100000 (above the I/O Memory "hole") leaving all physical addresses
below unused, don't perform phys<->mach mapping for addresses below 0x100000
or beyond the real hardware's physical memory.

-> /dev/mem works now as expected and X works in domain0.

Make bus_space map machine addresses instead of physical addresses.

branches: 1.1.2;
Add port to the Xen virtual machine monitor.
(see http://www.cl.cam.ac.uk/Research/SRG/netos/xen/)

A few changes:
* Use markers to reduce false sharing.
* Remove XENDEBUG_SYNC and several debug messages, they are just useless.
* Remove xen_vcpu_*. They are unused and not optimized: if we really
wanted to flush ranges we should pack the VAs in a mmuext_op array
instead of performing several hypercalls in a loop.
* Start removing PG_k.
* KNF, reorder, simplify and remove stupid comments.

Use the hypervisor to copy/zero pages. This saves us the extra overheads
of setting up temporary kernel mapping/unmapping.

riz@ reports savings of about 2s on a 120s kernel build.

branches: 1.37.2; 1.37.4; 1.37.12;
Extend the xpmap API, as described in [1]. This change is mechanical and
avoids exposing the MD phys_to_machine/machine_to_phys tables directly.
Added:

- xpmap_ptom handles PFN (pseudo physical) to MFN (machine frame number)
translations, and is under control of the domain.
- xpmap_mtop is its counterpart (MFN to PFN), and is under control of
hypervisor.

xpmap_ptom_map() map a pseudo-phys address to a machine address
xpmap_ptom_unmap() unmap a pseudo-phys address (invalidation)
xpmap_ptom_isvalid() check for pseudo-phys address validity

The parameters are physical/machine addresses, like bus_dma/bus_space(9).
As x86 MFNs are tracked by u_long (Xen's choice) while machine addresses
can be 64 bits entities (PAE), use ptoa() to avoid truncation when bit
shifting by PAGE_SHIFT.

I kept the same namespace (xpmap_) to avoid code churn.

[1] http://mail-index.netbsd.org/port-xen/2009/05/09/msg004951.html

XXX will document ptoa/atop/trunc_page separately.

Retire XEN_COMPAT_030001 as detailed on port-xen@:

http://mail-index.netbsd.org/port-xen/2012/06/25/msg007431.html

The xen_p2m API comes next.

ok bouyer@.
Tested on i386 PAE and amd64 (Xen 3.3 on private test bed, and
Xen 3.4 for Amazon EC2).

FWIW, Amazon always reported:

hypervisor0 at mainbus0: Xen version 3.4.3-kaos_t1micro

multiple times for Europe and US West-1, so I guess they are now at
3.4 (32 and 64 bits).

Enable the map/unmap recursive mapping functions for all Xen ports for
save/restore.

For an unknown reason (to me) Xen refuses to update VM translations
when the entry is pointing back to itself (which is precisely
what our recursive VM model does). So enable the functions that take
care of this, which will avoid all sort of memory corruption upon restore
leading domU to trample upon itself.

Save/restore works again for amd64. The occasional domU frontend corruption is
still present, but is harmless to dom0. Now we have a working shell and
ddb inside domU, that helps debugging a tiny bit.

XXX pull-up to -6.

- Convert x86 MD code, mainly pmap(9) e.g. TLB shootdown code, to use
kcpuset(9) and thus replace hardcoded CPU bitmasks. This removes the
limitation of maximum CPUs.

- Support up to 256 CPUs on amd64 architecture by default.

Bug fixes, improvements, completion of Xen part and testing on 64-core
AMD Opteron(tm) Processor 6282 SE (also, as Xen HVM domU with 128 CPUs)
by Manuel Bouyer.

branches: 1.33.2;
per-cpu shadow directory pages should be updated locally via cross-calls. Do this.

branches: 1.32.2;
Move Xen-specific functions to Xen pmap. Requested by cherry@.

Un'ifdef XEN in xen_pmap.c, it is always defined there.

Expose the PG_k #define pt/pd bit to both xen and "baremetal" x86. This is required, since kernel pages are mapped with user permissions in XEN/amd64 since the VM kernel runs in ring3. Since XEN/i386(including PAE) runs in ring1, supervisor mode is appropriate for these ports. We need to share this since the pmap implementation is still shared. Once the xen implementation is sufficiently independant of the x86 one, this can be made private to xen/include/xenpmap.h

[merging from cherry-xenmp] Make the xen MMU op queue locking api private. Implement per-cpu queues.

branches: 1.29.2;
remove unnecessary locking overhead for UP

Introduce locking primitives for Xen pte operations, and xen helper calls for MP related MMU ops

branches: 1.27.2;
Apply DRY: xpmap_{mtop,ptom}() can reuse xpmap_{mtop,ptom}_masked() for
the frame number lookup.

No functional change.

apply some _KERNEL_OPT.

Use only one function to pin pages with Xen, and provide macros to
call it for different levels (L1 => L4).

Replace all calls to xpq_queue_pin_table(...) in MD code with these new
functions, with proper #ifdef'ing depending on $MACHINE.

Rationale:
- only one function to modify for logging
- pushes responsibility to caller for chosing the proper pin level, rather
than Xen internal functions; this makes the pin level explicit rather than
implicit.

Boot tested for dom0 i386/amd64, PAE included. No functional change intended.

branches: 1.24.4; 1.24.6; 1.24.8;
Remove 3rd and 4th clauses. OK cl@ (copyright holder).

remove Xen2 support.
ok bouyer@

More i386PAE fixes:
- x86_round_page, x86_trunc_page, x86_btop and x86_ptob macros are used with
physical addresses; cast to paddr_t instead of u_long. Issue pointed out
by jym@
- machine_to_phys_mapping[] is a long. This is fine as it holds page
frame numbers (and this fits in a 32bit int as physical addresses are
only 36bits), but cast to paddr_t before << PAGE_SHIFT
- xen_start_info.store_mfn is a long; cast it to paddr_t before << PAGE_SHIFT.
should fix issue pointed out by cegger@

branches: 1.21.2; 1.21.4; 1.21.8; 1.21.12;
- add mfn_to_pfn() and pfn_to_mfn() macros, for rapid conversion between
pseudo-physical and machine frame numbers.

- add HYPERVISOR_crash() for i386 and amd64. Intended to be used by a domain
to notify Xen that it crashed on purpose, and request a dump (if applicable).

No functional changes intended.

Reviewed by Christoph (cegger@).

- printf -> aprint_*
- fix and add comments
- make some panic/error messages more relevant
- remove last '\n' in DPRINTK() macros, not required as it is already part of format string.

No functional changes.

branches: 1.19.6; 1.19.10; 1.19.16;
Merge the bouyer-xeni386 branch. This brings in PAE support to NetBSD xeni386
(domU only). PAE support is enabled by 'options PAE', see the new XEN3PAE_DOMU
and INSTALL_XEN3PAE_DOMU kernel config files.

See the comments in arch/i386/include/{pte.h,pmap.h} to see how it works.
In short, we still handle it as a 2-level MMU, with the second level page
directory being 4 pages in size. pmap switching is done by switching the
L2 pages in the L3 entries, instead of loading %cr3. This is almost required
by Xen, which handle the last L2 page (the one mapping 0xc0000000 - 0xffffffff)
in a very special way. But this approach should also work for native PAE
support if ever supported (in fact, the pmap should almost suport native
PAE, what's missing is bootstrap code in locore.S).

Merge the bouyer-xeni386 branch to head, at tag bouyer-xeni386-merge1 (the
branch is still active and will see i386PAE support developement).
Sumary of changes:
- switch xeni386 to the x86/x86/pmap.c, and the xen/x86/x86_xpmap.c
pmap bootstrap.
- merge back most of xen/i386/ to i386/i386
- change the build to reduce diffs between i386 and amd64 in file locations
- remove include files that were identical to the i386/amd64 counterparts,
the build will find them via the xen-ma/machine link.

branches: 1.17.6;
Use the new atomic ops.

Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.

branches: 1.15.8; 1.15.14; 1.15.26; 1.15.28; 1.15.32; 1.15.34;
Add ELF_PADDR_OFFSET and VIRT_ENTRY strings to __xen_guest ELF section,
so that our kernels works with newer xen-3 hypervisors; and correct the value
of VIRT_BASE for dom0.
Now that we can embed the values of KERNBASE and KERNTEXTOFF in the binary
for Xen, make the domU memory layout the same as dom0 for Xen3 (making
it the other way round doens't work; probably because of alignement
constraints in the hypervisor). The old domU layout is used if options
XEN_COMPAT_030001 is present in the kernel config file. Enable this the
domU kernel config files for now, in case someone wants to run a NetBSD
domU on an older Xen3 installation.

branches: 1.14.12; 1.14.14;
Implement MULTI_update_va_mapping() and MULTI_update_va_mapping_otherdomain(),
which fills in multicall arguments for __HYPERVISOR_update_va_mapping
and __HYPERVISOR_update_va_mapping_otherdomain dealing with
differences between i386 and amd64.

branches: 1.13.2;
Change "inline" back to "__inline" in .h files -- C99 is still too
new, and some apps compile things in C89 mode. C89 keywords stay.

As per core@.

branches: 1.12.2; 1.12.4;
Snapshot of work in progress on NetBSD port to Xen3:
- kernel (both dom0 and domU) boot, console is functionnal and it can starts
software from a ramdisk
- there is no driver front-end expect console for domU yet.
- dom0 can probe devices and ex(4) work when Xen3 is booted without acpi
and apic support. But the on-board IDE doens't get interrupts.
The PCI code still needs work (it's hardcoded to mode 1). Some of this
code should be shared with ../x86
The physical insterrupt code needs to get MPBIOS and ACPI support, and
do interrupt routing to properly interract with Xen.
To enable Xen-3.0 support, add
options XEN3
to your kernel config file (this will disable Xen2 support)
Changes affecting Xen-2.0 support (no functionnal changes intended):
- get more constants from genassym for assembly code
- remove some unneeded registers move from start()
- map the shared info page from start(), and remove the pte = 0xffffffff hack
- vector.S: in hypervisor_callback() make sure %esi points to
HYPERVISOR_shared_info before accessing the info page. Remplace some
hand-written assembly with the equivalent macro defined in frameasm.h
- more debug code, dissabled by default.

while here added my copyright on some files I worked on in 2005.

branches: 1.11.2;
Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.

merge ktrace-lwp.

some assym cleanup.
- move copyin and friends from locore.S to their own file, copy.S.
share it between i386 and xen.
- defparam KERNBASE and kill KERNBASE_LOCORE hack.
- add more symbols to assym.h and use it where appropriate.

Implement xpq_queue_machphys_update(), which queues a request to
update the machine to physical table (to be used after a
MEMOP_increase_reservation).

branches: 1.7.2;
avoid variable shadowing.

Always call the xpq_queue*() functions at splvm(), so that it's safe to call
them from interrupt context.
xpq_flush_queue() is called from IPL_NET in if_xennet.c, and
other xpq_queue* functions may be called from interrupt context via
pmap_kenter*(). Should fix port-xen/30153.
Thanks to Jason Thorpe and YAMAMOTO Takashi for enlightments on this issue.

s/foreing/foreign/g
ok'ed by Manuel Bouyer.

branches: 1.4.2;
Merge the bouyer-xen2 branch. This add supports for the Xen 2.0 virtual
machine kernel (both privileged and non-privileged domains), and remove support
for the old xen 1.2.

branches: 1.3.2; 1.3.6; 1.3.8; 1.3.10;
Rework the physical<->machine memory mapping: offset physical addresses
by 0x100000 (above the I/O Memory "hole") leaving all physical addresses
below unused, don't perform phys<->mach mapping for addresses below 0x100000
or beyond the real hardware's physical memory.

-> /dev/mem works now as expected and X works in domain0.

Make bus_space map machine addresses instead of physical addresses.

branches: 1.1.2;
Add port to the Xen virtual machine monitor.
(see http://www.cl.cam.ac.uk/Research/SRG/netos/xen/)