x86/xen: use correct printf specifier.

Use %p instead of wrongly casting to uintptr_t and printing with %lx.

Reported by: bapt
Fixes: 9a687d1fe3af ('x86/xen: introduce a Xen early init function')
Sponsored by: Cloud Software Group

x86/xen: implement early init hook

Unify the HVM and PVH early setup, byt making both rely on the hypervisor
initialization hook part of identify_hypervisor().

The current initialization takes care of the hypercall page, the sahred info
page and does any fixup necessary to metadata video console information if
FreeBSD is booted as the initial domain (so the video console is handed from
Xen into FreeBSD).

Note this has the nice side effect of also allowing to use the Xen console on
HVM guests, which allows to get rid of the QEMU emulated uart and still get
a nice text console.

Sponsored by: Cloud Software Group
Reviewed by: markj
Differential revision: https://reviews.freebsd.org/D43764

x86/xen: replace xen_cpuid_base with hv_base

Where possible, replace the usage of xen_cpuid_base for hv_base in preparation
for removing xen_cpuid_base.

No functional change intended.

Sponsored by: Cloud Software Group
Reviewed by: markj
Differential revision: https://reviews.freebsd.org/D43935

x86/xen: do video console fixup as part of early initialization

When FreeBSD is running as dom0 the video console metadata provided by the
bootloader might not be accurate, as Xen has very likely taken over the console
and possibly changed the mode.

Adjust the video console information in the kernel metadata as part of early
Xen initialization.

Sponsored by: Cloud Software Group
Reviewed by: imp
Differential revision: https://reviews.freebsd.org/D43934

x86/xen: move shared page setup to early init handler

As done with the hypercall page, move the setup fo the shared info page into
the newly introduced helper, which the aim of having a single helper and call
site used by both HVM and PV in order to setup the basic Xen environment.

Sponsored by: Cloud Software Group
Reviewed by: markj
Differential revision: https://reviews.freebsd.org/D43933

x86/xen: introduce a Xen early init function

Start by moving the hyeprcall setup to such function.

The aim is to have a function that does all the required Xen early
initialization for both HVM and PVH, instead of having it scattered across
different paths.

Sponsored by: Cloud Software Group
Reviewed by: markj
Differential revision: https://reviews.freebsd.org/D43932

x86/xen: introduce non-hypercall based emergency print

The current xc_printf() function uses an hypercall in order to send character
buffers to the hypervisor for it to print on the hypervisor console (if the
hypervisor is configured to print such messages).

This requires the hypercall page to be initialized, which is extra work and can
go wrong.

On x86 instead of using the console IO hypercall use the debug console IO port,
also called "port E9 hack". This allows sending characters to Xen using an
outb instruction, without any initialization required.

Keep the previous hypervisor based implementation by using the weak attribute,
which allows each architecture to provide an alternate (arch-specific)
implementation.

Sponsored by: Cloud Software Group
Reviewed by: markj
Differential revision: https://reviews.freebsd.org/D43929

x86/xen: fix HVM guest hypercall page setup

c7368ccb6801 didn't take into account that vm_guest will also get setup by
generic identify CPU code, and hence by the time xen_hvm_init() gets called
vm_guest will always be set if running as a Xen guest, either by the PVH entry
point code, or by generic CPU identification.

xen_hvm_init() and xen_hvm_init_hypercall_stubs() were relying on xen_domain()
returning false when running as an HVM guest, and used that into order to
figure out whether hypercall page needed to be populated.

Get rid of such assumptions and simplify the code since legacy PVH is no
longer supported.

This fixes booting FreeBSD as a Xen HVM guest.

Fixes: c7368ccb6801 ('xen: remove xen_domain_type enum/variable')
Sponsored by: Cloud Software Group

xen: remove xen_domain_type enum/variable

The vm_guest variable readily covers all uses of xen_domain_type, so
merge them together. Since support for PV domains has been removed
hard-core xen_pv_domain() to return false.

Reviewed by: royger

sys: Automated cleanup of cdefs and other formatting

Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.

Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/

Sponsored by: Netflix

sys: Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

xen: move vcpu_info to common, leave hook for setup

vcpu_info is crucial for the Xen event channel core. Since both the
data and setup steps are identical between architectures, move them to
the common file. Since there is no cross-architecture method to call
a function on every processor during bring-up, simply leave the setup
function.

The number of vcpu_info structures available on the shared information
page varies by architecture. Instead of hard-coding the count use
nitems(). Add a warning message for this being used.

Switch to XEN_VCPUID() and use Xen's typedefs.

panic() on failure since >32 processors is no longer unusual.

royger: Specify 64-byte alignment for vcpu_info to try to defend
against vcpu_info crossing a page boundary. Add detection for this
limit.

Reviewed by: royger

spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD

The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix

xen: move common variables off of sys/x86/xen/hvm.c

The xen_domain_type and HYPERVISOR_shared_info variables are shared by
all Xen architectures, so they should be in common rather than
reimplemented by each architecture.

hvm_start_flags is used by xen_initial_domain() and so needs to be in
common.

Reviewed by: royger
Differential Revision: https://reviews.freebsd.org/D28982

Fix various places which cast a pointer to a vm_paddr_t or vice versa.

GCC warns about the mismatched sizes on i386 where vm_paddr_t is 64
bits.

Reviewed by: imp, markj
Differential Revision: https://reviews.freebsd.org/D36750

xen/blkback: do not use x86 CPUID in generic code

Move checker for whether Xen creates IOMMU mappings for foreign pages
into a helper that's defined in arch-specific code.

Reported by: Elliott Mitchell <ehem+freebsd@m5p.com>
Fixes: 1d528f95e8ce ('xen/blkback: remove bounce buffering mode')
Sponsored by: Citrix Systems R&D

xen: switch to use headers in contrib

These headers originate with the Xen project and shouldn't be mixed with
the main portion of the FreeBSD kernel. Notably they shouldn't be the
target of clean-up commits.

Switch to use the headers in sys/contrib/xen.

Reviewed by: royger

x86/xen: use x{2}APIC if virtualized by hardware

Instead of using event channels or hypercalls to deal with IPIs and
NMIs.

Using a hardware virtualized APIC should be faster than using any PV
interface, since the VM exit can be avoided.

Xen exposes whether the domain is using hardware assisted x{2}APIC
emulation in a CPUID bit.

Sponsored by: Citrix Systems R&D

xen: remove hypervisor_info

This was a source of indirection needed to support PVHv1. Now that that
support has been removed, we can eliminate it.

Reviewed by: royger

x86: clean up empty lines in .c and .h files

xen: introduce a new way to setup event channel upcall

The main differences with the currently implemented method are:

- Requires a local APIC EOI, since it doesn't bypass the local APIC
as the previous method used to do.
- Can be set to use different IDT vectors on each vCPU. Note that
FreeBSD doesn't make use of this feature since the event channel
IDT vector is reserved system wide.

Note that the old method of setting the event channel upcall is
not removed, and will be used as a fallback if this newly introduced
method is not available.

MFC after: 1 month
Sponsored by: Citrix Systems R&D

xen: fix setting legacy PVH vcpu id

The recommended way to obtain the vcpu id is using the cpuid
instruction with a specific leaf value. This leaf value must be
obtained at runtime, and it's done when populating the hypercall page.

Legacy PVH however will get the hypercall page populated by the
hypervisor itself before booting, so the cpuid leaf was not actually
set, thus preventing setting the vcpu id value from cpuid.

Fix this by making sure the cpuid leaf has been probed before
attempting to set the vcpu id.

Approved by: re (gjb)
Sponsored by: Citrix Systems R&D

xen: remove xenpv_set_ids

The vcpu_id for legacy PVH mode can be set from the output of cpuid,
so there's no need to have a special function to set it.

Also note that xenpv_set_ids should have been executed only for PV
guests, but was executed for all guests types and vcpu_id was later
fixed up for HVM guests.

Reported by: cperciva
Approved by: re (gjb)
Sponsored by: Citrix Systems R&D

xen: implement early init helper for PVHv2

In order to setup an initial environment and jump into the generic
hammer_time initialization function. Some of the code is shared with
PVHv1, while other code is PVHv2 specific.

This allows booting FreeBSD as a PVHv2 DomU and Dom0.

Sponsored by: Citrix Systems R&D

xen: allow very early initialization of the hypercall page

Allow the hypercall page to be initialized very early, even before
vtophys is functional. Also make the function global so it can be
called by other files.

This will be needed in order to perform the early bringup on PVHv2
guests.

Sponsored by: Citrix Systems R&D

xen: remove direct usage of HYPERVISOR_start_info

HYPERVISOR_start_info is only available to PV and PVHv1 guests, HVM
and PVHv2 guests get this data from HVM parameters that are fetched
using a hypercall.

Instead provide a set of helper functions that should be used to fetch
this data. The helper functions have different implementations
depending on whether FreeBSD is running as PVHv1 or HVM/PVHv2 guest
type.

This helps to cleanup generic Xen code by removing quite a lot of
xen_pv_domain and xen_hvm_domain macro usages.

Sponsored by: Citrix Systems R&D

xen: obtain vCPU ID from CPUID

The Xen vCPU ID can be fetched from the cpuid instead of inferring it
from the ACPI ID.

Sponsored by: Citrix Systems R&D

xen: limit the number of hypercall pages to 1

The interface already guarantees that the number of hypercall pages is
always going to be 1, see the comment in interface/arch-x86/cpuid.h

Sponsored by: Citrix Systems R&D

sys/x86: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Fix build for !SMP kernels after the Xen MSIX workaround.

Move msix_disable_migration under #ifdef SMP since it doesn't make sense
for !SMP kernels.

PR: 212014
Reported by: Glyn Grinstead <glyn@grinstead.org>
MFC after: 3 days

xen: automatically disable MSI-X interrupt migration

If the hypervisor version is smaller than 4.6.0. Xen commits 74fd00 and
70a3cb are required on the hypervisor side for this to be fixed, and those
are only included in 4.6.0, so stay on the safe side and disable MSI-X
interrupt migration on anything older than 4.6.0.

It should not cause major performance degradation unless a lot of MSI-X
interrupts are allocated.

Sponsored by: Citrix Systems R&D
MFC after: 3 days
Reviewed by: jhb
Differential revision: https://reviews.freebsd.org/D7148

xen: fix dropping bitmap IPIs during resume

Current Xen resume code clears all pending bitmap IPIs on resume, which is
not correct. Instead re-inject bitmap IPI vectors on resume to all CPUs in
order to acknowledge any pending bitmap IPIs.

Sponsored by: Citrix Systems R&D
MFC after: 2 weeks

xen: allow disabling PV disks and nics

Introduce two new loader tunnables that can be used to disable PV disks and
PV nics at boot time. They default to 0 and should be set to 1 (or any
number different than 0) in order to disable the PV devices:

hw.xen.disable_pv_disks=1
hw.xen.disable_pv_nics=1

In /boot/loader.conf will disable both PV disks and nics.

Sponsored by: Citrix Systems R&D
Tested by: Karl Pielorz <kpielorz_lst@tdx.co.uk>
MFC after: 1 week

xen: implement the privcmd user-space device

This device is only attached to priviledged domains, and allows the
toolstack to interact with Xen. The two functions of the privcmd
interface is to allow the execution of hypercalls from user-space, and
the mapping of foreign domain memory.

Sponsored by: Citrix Systems R&D

i386/include/xen/hypercall.h:
amd64/include/xen/hypercall.h:
- Introduce a function to make generic hypercalls into Xen.

xen/interface/xen.h:
xen/interface/memory.h:
- Import the new hypercall XENMEM_add_to_physmap_range used by
auto-translated guests to map memory from foreign domains.

dev/xen/privcmd/privcmd.c:
- This device has the following functions:
- Allow user-space applications to make hypercalls into Xen.
- Allow user-space applications to map memory from foreign domains,
this is accomplished using the newly introduced hypercall
(XENMEM_add_to_physmap_range).

xen/privcmd.h:
- Public ioctl interface for the privcmd device.

x86/xen/hvm.c:
- Remove declaration of hypercall_page, now it's declared in
hypercall.h.

conf/files:
- Add the privcmd device to the build process.

xen: add hooks for Xen PV APIC

Create the necessary hooks in order to provide a Xen PV APIC
implementation that can be used on PVH. Most of the lapic ops
shouldn't be called on Xen, since we trap those operations at a higher
layer.

Sponsored by: Citrix Systems R&D
Approved by: gibbs

x86/xen/hvm.c:
x86/xen/xen_apic.c:
- Move IPI related code to xen_apic.c

x86/xen/xen_apic.c:
- Introduce Xen PV APIC implementation, most of the functions of the
lapic interface should never be called when running as PV(H) guest,
so make sure FreeBSD panics when trying to use one of those.
- Define the Xen APIC implementation in xen_apic_ops.

xen/xen_pv.h:
- Extern declaration of the xen_apic struct.

x86/xen/pv.c:
- Use xen_apic_ops as apic_ops when running as PVH guest.

conf/files.amd64:
conf/files.i386:
- Include the xen_apic.c file in the build of i386/amd64 kernels
using XENHVM.

amd64/i386: introduce APIC hooks for different APIC implementations.

This is needed for Xen PV(H) guests, since there's no hardware lapic
available on this kind of domains. This commit should not change
functionality.

Sponsored by: Citrix Systems R&D
Reviewed by: jhb
Approved by: gibbs

amd64/include/cpu.h:
amd64/amd64/mp_machdep.c:
i386/include/cpu.h:
i386/i386/mp_machdep.c:
- Remove lapic_ipi_vectored hook from cpu_ops, since it's now
implemented in the lapic hooks.

amd64/amd64/mp_machdep.c:
i386/i386/mp_machdep.c:
- Use lapic_ipi_vectored directly, since it's now an inline function
that will call the appropiate hook.

x86/x86/local_apic.c:
- Prefix bare metal public lapic functions with native_ and mark them
as static.
- Define default implementation of apic_ops.

x86/include/apicvar.h:
- Declare the apic_ops structure and create inline functions to
access the hooks, so the change is transparent to existing users of
the lapic_ functions.

x86/xen/hvm.c:
- Switch to use the new apic_ops.

Fix build without SMP.

PR: kern/187854
MFC after: 1 week

xen: changes to hvm code in order to support PVH guests

On PVH we don't need to init the shared info page, or disable emulated
devices. Also, make sure PV IPIs are set before starting the APs.

Approved by: gibbs
Sponsored by: Citrix Systems R&D

x86/xen/hvm.c:
- Return early from functions that are no-ops on Xen PVH guests.
- In order to make sure PV IPIs are setup before AP startup,
initialize them in SI_SUB_SMP-1.

xen: use the same hypercall mechanism for XEN and XENHVM

Currently XEN (PV) and XENHVM (PVHVM) ports use different ways to
issue hypercalls, unify this by filling the hypercall_page under HVM
also.

Approved by: gibbs
Sponsored by: Citrix Systems R&D

amd64/include/xen/hypercall.h:
- Unify Xen hypercall code by always using the PV way.

i386/i386/locore.s:
- Define hypercall_page on i386 XENHVM.

x86/xen/hvm.c:
- Fill hypercall_page on XENHVM kernels using the HVM method (only
when running as an HVM guest).

xen: add PV/PVH kernel entry point

Add the PV/PVH entry point and the low level functions for PVH
early initialization.

Approved by: gibbs
Sponsored by: Citrix Systems R&D

amd64/amd64/genassym.c:
- Add __FreeBSD_version define to assym.s so it can be used for the
Xen notes.

amd64/amd64/locore.S:
- Make bootstack global so it can be used from Xen kernel entry
point.

amd64/amd64/xen-locore.S:
- Add Xen notes to the kernel.
- Add the Xen PV entry point, that is going to call hammer_time_xen.

amd64/include/asmacros.h:
- Add ELFNOTE macros.

i386/xen/xen_machdep.c:
- Define HYPERVISOR_start_info for the XEN i386 PV port, which is
going to be used in some shared code between PV and PVH.

x86/xen/hvm.c:
- Define HYPERVISOR_start_info for the PVH port.

x86/xen/pv.c:
- Introduce hammer_time_xen which is going to perform early setup for
Xen PVH:
- Setup shared Xen variables start_info, shared_info and
xen_store.
- Set guest type.
- Create initial page tables as FreeBSD expects to find them.
- Call into native init function (hammer_time).

xen/xen-os.h:
- Declare HYPERVISOR_start_info.

conf/files.amd64:
- Add amd64/amd64/locore.S and x86/xen/pv.c to the list of files.

amd64/i386: switch IPI handlers to C code.

Move asm IPIs handlers to C code, so both Xen and native IPI handlers
share the same code.

Reviewed by: jhb
Approved by: gibbs
Sponsored by: Citrix Systems R&D

amd64/amd64/apic_vector.S:
i386/i386/apic_vector.s:
- Remove asm coded IPI handlers and instead call the newly introduced
C variants.

amd64/amd64/mp_machdep.c:
i386/i386/mp_machdep.c:
- Add C coded clones to the asm IPI handlers (moved from
x86/xen/hvm.c).

i386/include/smp.h:
amd64/include/smp.h:
- Add prototypes for the C IPI handlers.

x86/xen/hvm.c:
- Move the C IPI handlers to mp_machdep and call those in the Xen IPI
handlers.

i386/xen/mp_machdep.c:
- Add dummy IPI handlers to the i386 Xen PV port (this port doesn't
support SMP).

Formalize the concept of virtual CPU ids by adding a per-cpu vcpu_id
field. Perform vcpu enumeration for Xen PV and HVM environments
and convert all Xen drivers to use vcpu_id instead of a hard coded
assumption of the mapping algorithm (acpi or apic ID) in use.

Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
Reviewed by: gibbs
Approved by: re (blanket Xen)

amd64/include/pcpu.h:
i386/include/pcpu.h:
Add vcpu_id to the amd64 and i386 pcpu structures.

dev/xen/timer/timer.c
x86/xen/xen_intr.c
Use new vcpu_id instead of assuming acpi_id == vcpu_id.

i386/xen/mp_machdep.c:
i386/xen/mptable.c
x86/xen/hvm.c:
Perform Xen HVM and Xen full PV vcpu_id mapping.

x86/xen/hvm.c:
x86/acpica/madt.c
Change SYSINIT ordering of acpi CPU enumeration so that it
is guaranteed to be available at the time of Xen HVM vcpu
id mapping.

Correct panic caused by attaching both Xen PV and HyperV virtualization
aware drivers on Xen hypervisors that advertise support for some
HyperV features.

x86/xen/hvm.c:
When running in HVM mode on a Xen hypervisor, set vm_guest
to VM_GUEST_XEN so other virtualization aware components in
the FreeBSD kernel can detect this mode is active.

dev/hyperv/vmbus/hv_hv.c:
Use vm_guest to ignore Xen's HyperV emulation when Xen is
detected and Xen PV drivers are active.

Reported by: Shanker Balan
Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
Reviewed by: gibbs
Approved by: re (Xen blanket)

sys/x86/xen/hvm.c:
Set cpu_ops correctly for Xen hypervisors lacking the
vector callback feature.

Set preliminary Xen cpu_ops settings during early HVM
initialization. The old location raced with the startup
of APs.

Submitted by: Roger Pau Monné
Reviewed by: gibbs
Approved by: re (blanket Xen)

Merge Xen PVHVM support into the GENERIC kernel config for both
amd64 and i386.

Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
Reviewed by: gibbs
Approved by: re (blanket Xen)
MFC after: 2 weeks

sys/amd64/amd64/mp_machdep.c:
sys/amd64/include/cpu.h:
sys/i386/i386/mp_machdep.c:
sys/i386/include/cpu.h:
- Introduce two new CPU hooks for initialization and resume
purposes. This allows us to get rid of the XENHVM ifdefs in
mp_machdep, and also sets some hooks into common code that can be
used by other hypervisor implementations.

sys/amd64/conf/XENHVM:
sys/i386/conf/XENHVM:
- Remove these configs now that GENERIC has builtin support for Xen
HVM.

sys/kern/subr_smp.c:
- Make sure there are no pending IPIs when suspending a system.

sys/x86/xen/hvm.c:
- Add cpu init and resume vectors that are called from mp_machdep
using the new hooks.
- Only clear the vcpu_info mapping data on resume. It is already
clear for the BSP on a cold boot and is set correctly as APs
are started.
- Gate xen_hvm_init_cpu only to systems running under Xen.

sys/x86/xen/xen_intr.c:
- Gate the setup of event channels only to systems running under Xen.

Add support for suspend/resume/migration operations when running as a
Xen PVHVM guest.

Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
Reviewed by: gibbs
Approved by: re (blanket Xen)
MFC after: 2 weeks

sys/amd64/amd64/mp_machdep.c:
sys/i386/i386/mp_machdep.c:
- Make sure that are no MMU related IPIs pending on migration.
- Reset pending IPI_BITMAP on resume.
- Init vcpu_info on resume.

sys/amd64/include/intr_machdep.h:
sys/i386/include/intr_machdep.h:
sys/x86/acpica/acpi_wakeup.c:
sys/x86/x86/intr_machdep.c:
sys/x86/isa/atpic.c:
sys/x86/x86/io_apic.c:
sys/x86/x86/local_apic.c:
- Add a "suspend_cancelled" parameter to pic_resume(). For the
Xen PIC, restoration of interrupt services differs between
the aborted suspend and normal resume cases, so we must provide
this information.

sys/dev/acpica/acpi_timer.c:
sys/dev/xen/timer/timer.c:
sys/timetc.h:
- Don't swap out "suspend safe" timers across a suspend/resume
cycle. This includes the Xen PV and ACPI timers.

sys/dev/xen/control/control.c:
- Perform proper suspend/resume process for PVHVM:
- Suspend all APs before going into suspension, this allows us
to reset the vcpu_info on resume for each AP.
- Reset shared info page and callback on resume.

sys/dev/xen/timer/timer.c:
- Implement suspend/resume support for the PV timer. Since FreeBSD
doesn't perform a per-cpu resume of the timer, we need to call
smp_rendezvous in order to correctly resume the timer on each CPU.

sys/dev/xen/xenpci/xenpci.c:
- Don't reset the PCI interrupt on each suspend/resume.

sys/kern/subr_smp.c:
- When suspending a PVHVM domain make sure there are no MMU IPIs
in-flight, or we will get a lockup on resume due to the fact that
pending event channels are not carried over on migration.
- Implement a generic version of restart_cpus that can be used by
suspended and stopped cpus.

sys/x86/xen/hvm.c:
- Implement resume support for the hypercall page and shared info.
- Clear vcpu_info so it can be reset by APs when resuming from
suspension.

sys/dev/xen/xenpci/xenpci.c:
sys/x86/xen/hvm.c:
sys/x86/xen/xen_intr.c:
- Support UP kernel configurations.

sys/x86/xen/xen_intr.c:
- Properly rebind per-cpus VIRQs and IPIs on resume.

Implement PV IPIs for PVHVM guests and further converge PV and HVM
IPI implmementations.

Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D
Submitted by: gibbs (misc cleanup, table driven config)
Reviewed by: gibbs
MFC after: 2 weeks

sys/amd64/include/cpufunc.h:
sys/amd64/amd64/pmap.c:
Move invltlb_globpcid() into cpufunc.h so that it can be
used by the Xen HVM version of tlb shootdown IPI handlers.

sys/x86/xen/xen_intr.c:
sys/xen/xen_intr.h:
Rename xen_intr_bind_ipi() to xen_intr_alloc_and_bind_ipi(),
and remove the ipi vector parameter. This api allocates
an event channel port that can be used for ipi services,
but knows nothing of the actual ipi for which that port
will be used. Removing the unused argument and cleaning
up the comments surrounding its declaration helps clarify
its actual role.

sys/amd64/amd64/mp_machdep.c:
sys/amd64/include/cpu.h:
sys/i386/i386/mp_machdep.c:
sys/i386/include/cpu.h:
Implement a generic framework for amd64 and i386 that allows
the implementation of certain CPU management functions to
be selected at runtime. Currently this is only used for
the ipi send function, which we optimize for Xen when running
on a Xen hypervisor, but can easily be expanded to support
more operations.

sys/x86/xen/hvm.c:
Implement Xen PV IPI handlers and operations, replacing native
send IPI.

sys/amd64/include/pcpu.h:
sys/i386/include/pcpu.h:
sys/i386/include/smp.h:
Remove NR_VIRQS and NR_IPIS from FreeBSD headers. NR_VIRQS
is defined already for us in the xen interface files.
NR_IPIS is only needed in one file per Xen platform and is
easily inferred by the IPI vector table that is defined in
those files.

sys/i386/xen/mp_machdep.c:
Restructure to more closely match the HVM implementation by
performing table driven IPI setup.

Conform to style(9). No functional changes.

sys/x86/xen/hvm.c:
Do not rely on implicit conversion to boolean in expressions
(e.g. use "if (rc != 0)" instead of "if (rc)".

Line continuations for functions are indented an additional
4 spaces.

Insert an empty line if the function has no local variables.

Prefer separate initializtion statements to initialzing
local variables in their declaration.

Braces that are not necessary may be left out.

MFC after: 2 weeks

Implement vector callback for PVHVM and unify event channel implementations

Re-structure Xen HVM support so that:
- Xen is detected and hypercalls can be performed very
early in system startup.
- Xen interrupt services are implemented using FreeBSD's native
interrupt delivery infrastructure.
- the Xen interrupt service implementation is shared between PV
and HVM guests.
- Xen interrupt handlers can optionally use a filter handler
in order to avoid the overhead of dispatch to an interrupt
thread.
- interrupt load can be distributed among all available CPUs.
- the overhead of accessing the emulated local and I/O apics
on HVM is removed for event channel port events.
- a similar optimization can eventually, and fairly easily,
be used to optimize MSI.

Early Xen detection, HVM refactoring, PVHVM interrupt infrastructure,
and misc Xen cleanups:

Sponsored by: Spectra Logic Corporation

Unification of PV & HVM interrupt infrastructure, bug fixes,
and misc Xen cleanups:

Submitted by: Roger Pau Monné
Sponsored by: Citrix Systems R&D

sys/x86/x86/local_apic.c:
sys/amd64/include/apicvar.h:
sys/i386/include/apicvar.h:
sys/amd64/amd64/apic_vector.S:
sys/i386/i386/apic_vector.s:
sys/amd64/amd64/machdep.c:
sys/i386/i386/machdep.c:
sys/i386/xen/exception.s:
sys/x86/include/segments.h:
Reserve IDT vector 0x93 for the Xen event channel upcall
interrupt handler. On Hypervisors that support the direct
vector callback feature, we can request that this vector be
called directly by an injected HVM interrupt event, instead
of a simulated PCI interrupt on the Xen platform PCI device.
This avoids all of the overhead of dealing with the emulated
I/O APIC and local APIC. It also means that the Hypervisor
can inject these events on any CPU, allowing upcalls for
different ports to be handled in parallel.

sys/amd64/amd64/mp_machdep.c:
sys/i386/i386/mp_machdep.c:
Map Xen per-vcpu area during AP startup.

sys/amd64/include/intr_machdep.h:
sys/i386/include/intr_machdep.h:
Increase the FreeBSD IRQ vector table to include space
for event channel interrupt sources.

sys/amd64/include/pcpu.h:
sys/i386/include/pcpu.h:
Remove Xen HVM per-cpu variable data. These fields are now
allocated via the dynamic per-cpu scheme. See xen_intr.c
for details.

sys/amd64/include/xen/hypercall.h:
sys/dev/xen/blkback/blkback.c:
sys/i386/include/xen/xenvar.h:
sys/i386/xen/clock.c:
sys/i386/xen/xen_machdep.c:
sys/xen/gnttab.c:
Prefer FreeBSD primatives to Linux ones in Xen support code.

sys/amd64/include/xen/xen-os.h:
sys/i386/include/xen/xen-os.h:
sys/xen/xen-os.h:
sys/dev/xen/balloon/balloon.c:
sys/dev/xen/blkback/blkback.c:
sys/dev/xen/blkfront/blkfront.c:
sys/dev/xen/console/xencons_ring.c:
sys/dev/xen/control/control.c:
sys/dev/xen/netback/netback.c:
sys/dev/xen/netfront/netfront.c:
sys/dev/xen/xenpci/xenpci.c:
sys/i386/i386/machdep.c:
sys/i386/include/pmap.h:
sys/i386/include/xen/xenfunc.h:
sys/i386/isa/npx.c:
sys/i386/xen/clock.c:
sys/i386/xen/mp_machdep.c:
sys/i386/xen/mptable.c:
sys/i386/xen/xen_clock_util.c:
sys/i386/xen/xen_machdep.c:
sys/i386/xen/xen_rtc.c:
sys/xen/evtchn/evtchn_dev.c:
sys/xen/features.c:
sys/xen/gnttab.c:
sys/xen/gnttab.h:
sys/xen/hvm.h:
sys/xen/xenbus/xenbus.c:
sys/xen/xenbus/xenbus_if.m:
sys/xen/xenbus/xenbusb_front.c:
sys/xen/xenbus/xenbusvar.h:
sys/xen/xenstore/xenstore.c:
sys/xen/xenstore/xenstore_dev.c:
sys/xen/xenstore/xenstorevar.h:
Pull common Xen OS support functions/settings into xen/xen-os.h.

sys/amd64/include/xen/xen-os.h:
sys/i386/include/xen/xen-os.h:
sys/xen/xen-os.h:
Remove constants, macros, and functions unused in FreeBSD's Xen
support.

sys/xen/xen-os.h:
sys/i386/xen/xen_machdep.c:
sys/x86/xen/hvm.c:
Introduce new functions xen_domain(), xen_pv_domain(), and
xen_hvm_domain(). These are used in favor of #ifdefs so that
FreeBSD can dynamically detect and adapt to the presence of
a hypervisor. The goal is to have an HVM optimized GENERIC,
but more is necessary before this is possible.

sys/amd64/amd64/machdep.c:
sys/dev/xen/xenpci/xenpcivar.h:
sys/dev/xen/xenpci/xenpci.c:
sys/x86/xen/hvm.c:
sys/sys/kernel.h:
Refactor magic ioport, Hypercall table and Hypervisor shared
information page setup, and move it to a dedicated HVM support
module.

HVM mode initialization is now triggered during the
SI_SUB_HYPERVISOR phase of system startup. This currently
occurs just after the kernel VM is fully setup which is
just enough infrastructure to allow the hypercall table
and shared info page to be properly mapped.

sys/xen/hvm.h:
sys/x86/xen/hvm.c:
Add definitions and a method for configuring Hypervisor event
delievery via a direct vector callback.

sys/amd64/include/xen/xen-os.h:
sys/x86/xen/hvm.c:

sys/conf/files:
sys/conf/files.amd64:
sys/conf/files.i386:
Adjust kernel build to reflect the refactoring of early
Xen startup code and Xen interrupt services.

sys/dev/xen/blkback/blkback.c:
sys/dev/xen/blkfront/blkfront.c:
sys/dev/xen/blkfront/block.h:
sys/dev/xen/control/control.c:
sys/dev/xen/evtchn/evtchn_dev.c:
sys/dev/xen/netback/netback.c:
sys/dev/xen/netfront/netfront.c:
sys/xen/xenstore/xenstore.c:
sys/xen/evtchn/evtchn_dev.c:
sys/dev/xen/console/console.c:
sys/dev/xen/console/xencons_ring.c
Adjust drivers to use new xen_intr_*() API.

sys/dev/xen/blkback/blkback.c:
Since blkback defers all event handling to a taskqueue,
convert this task queue to a "fast" taskqueue, and schedule
it via an interrupt filter. This avoids an unnecessary
ithread context switch.

sys/xen/xenstore/xenstore.c:
The xenstore driver is MPSAFE. Indicate as much when
registering its interrupt handler.

sys/xen/xenbus/xenbus.c:
sys/xen/xenbus/xenbusvar.h:
Remove unused event channel APIs.

sys/xen/evtchn.h:
Remove all kernel Xen interrupt service API definitions
from this file. It is now only used for structure and
ioctl definitions related to the event channel userland
device driver.

Update the definitions in this file to match those from
NetBSD. Implementing this interface will be necessary for
Dom0 support.

sys/xen/evtchn/evtchnvar.h:
Add a header file for implemenation internal APIs related
to managing event channels event delivery. This is used
to allow, for example, the event channel userland device
driver to access low-level routines that typical kernel
consumers of event channel services should never access.

sys/xen/interface/event_channel.h:
sys/xen/xen_intr.h:
Standardize on the evtchn_port_t type for referring to
an event channel port id. In order to prevent low-level
event channel APIs from leaking to kernel consumers who
should not have access to this data, the type is defined
twice: Once in the Xen provided event_channel.h, and again
in xen/xen_intr.h. The double declaration is protected by
__XEN_EVTCHN_PORT_DEFINED__ to ensure it is never declared
twice within a given compilation unit.

sys/xen/xen_intr.h:
sys/xen/evtchn/evtchn.c:
sys/x86/xen/xen_intr.c:
sys/dev/xen/xenpci/evtchn.c:
sys/dev/xen/xenpci/xenpcivar.h:
New implementation of Xen interrupt services. This is
similar in many respects to the i386 PV implementation with
the exception that events for bound to event channel ports
(i.e. not IPI, virtual IRQ, or physical IRQ) are further
optimized to avoid mask/unmask operations that aren't
necessary for these edge triggered events.

Stubs exist for supporting physical IRQ binding, but will
need additional work before this implementation can be
fully shared between PV and HVM.

sys/amd64/amd64/mp_machdep.c:
sys/i386/i386/mp_machdep.c:
sys/i386/xen/mp_machdep.c
sys/x86/xen/hvm.c:
Add support for placing vcpu_info into an arbritary memory
page instead of using HYPERVISOR_shared_info->vcpu_info.
This allows the creation of domains with more than 32 vcpus.

sys/i386/i386/machdep.c:
sys/i386/xen/clock.c:
sys/i386/xen/xen_machdep.c:
sys/i386/xen/exception.s:
Add support for new event channle implementation.