arm/xen: fix xen_vcpu_info allocation alignment

xen_vcpu_info is a percpu area than needs to be mapped by Xen.
Currently, it could cross a page boundary resulting in Xen being unable
to map it:

[ 0.567318] kernel BUG at arch/arm64/xen/../../arm/xen/enlighten.c:164!
[ 0.574002] Internal error: Oops - BUG: 00000000f2000800 [#1] PREEMPT SMP

Fix the issue by using __alloc_percpu and requesting alignment for the
memory allocation.

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>

Link: https://lore.kernel.org/r/alpine.DEB.2.22.394.2311221501340.2053963@ubuntu-linux-20-04-desktop
Fixes: 24d5373dda7c ("arm/xen: Use alloc_percpu rather than __alloc_percpu")
Reviewed-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Juergen Gross <jgross@suse.com>

arm64/arm: xen: enlighten: Fix KPTI checks

When KPTI is in use, we cannot register a runstate region as XEN
requires that this is always a valid VA, which we cannot guarantee. Due
to this, xen_starting_cpu() must avoid registering each CPU's runstate
region, and xen_guest_init() must avoid setting up features that depend
upon it.

We tried to ensure that in commit:

f88af7229f6f22ce (" xen/arm: do not setup the runstate info page if kpti is enabled")

... where we added checks for xen_kernel_unmapped_at_usr(), which wraps
arm64_kernel_unmapped_at_el0() on arm64 and is always false on 32-bit
arm.

Unfortunately, as xen_guest_init() is an early_initcall, this happens
before secondary CPUs are booted and arm64 has finalized the
ARM64_UNMAP_KERNEL_AT_EL0 cpucap which backs
arm64_kernel_unmapped_at_el0(), and so this can subsequently be set as
secondary CPUs are onlined. On a big.LITTLE system where the boot CPU
does not require KPTI but some secondary CPUs do, this will result in
xen_guest_init() intializing features that depend on the runstate
region, and xen_starting_cpu() registering the runstate region on some
CPUs before KPTI is subsequent enabled, resulting the the problems the
aforementioned commit tried to avoid.

Handle this more robsutly by deferring the initialization of the
runstate region until secondary CPUs have been initialized and the
ARM64_UNMAP_KERNEL_AT_EL0 cpucap has been finalized. The per-cpu work is
moved into a new hotplug starting function which is registered later
when we're certain that KPTI will not be used.

Fixes: f88af7229f6f ("xen/arm: do not setup the runstate info page if kpti is enabled")
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Bertrand Marquis <bertrand.marquis@arm.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>

xen: simplify evtchn_do_upcall() call maze

There are several functions involved for performing the functionality
of evtchn_do_upcall():

- __xen_evtchn_do_upcall() doing the real work
- xen_hvm_evtchn_do_upcall() just being a wrapper for
__xen_evtchn_do_upcall(), exposed for external callers
- xen_evtchn_do_upcall() calling __xen_evtchn_do_upcall(), too, but
without any user

Simplify this maze by:

- removing the unused xen_evtchn_do_upcall()
- removing xen_hvm_evtchn_do_upcall() as the only left caller of
__xen_evtchn_do_upcall(), while renaming __xen_evtchn_do_upcall() to
xen_evtchn_do_upcall()

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Juergen Gross <jgross@suse.com>

xen/virtio: Optimize the setup of "xen-grant-dma" devices

This is needed to avoid having to parse the same device-tree
several times for a given device.

For this to work we need to install the xen_virtio_restricted_mem_acc
callback in Arm's xen_guest_init() which is same callback as x86's
PV and HVM modes already use and remove the manual assignment in
xen_setup_dma_ops(). Also we need to split the code to initialize
backend_domid into a separate function.

Prior to current patch we parsed the device-tree three times:
1. xen_setup_dma_ops()->...->xen_is_dt_grant_dma_device()
2. xen_setup_dma_ops()->...->xen_dt_grant_init_backend_domid()
3. xen_virtio_mem_acc()->...->xen_is_dt_grant_dma_device()

With current patch we parse the device-tree only once in
xen_virtio_restricted_mem_acc()->...->xen_dt_grant_init_backend_domid()

Other benefits are:
- Not diverge from x86 when setting up Xen grant DMA ops
- Drop several global functions

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Xenia Ragiadakou <burzalodowa@gmail.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Link: https://lore.kernel.org/r/20221025162004.8501-2-olekstysh@gmail.com
Signed-off-by: Juergen Gross <jgross@suse.com>

xen: don't require virtio with grants for non-PV guests

Commit fa1f57421e0b ("xen/virtio: Enable restricted memory access using
Xen grant mappings") introduced a new requirement for using virtio
devices: the backend now needs to support the VIRTIO_F_ACCESS_PLATFORM
feature.

This is an undue requirement for non-PV guests, as those can be operated
with existing backends without any problem, as long as those backends
are running in dom0.

Per default allow virtio devices without grant support for non-PV
guests.

On Arm require VIRTIO_F_ACCESS_PLATFORM for devices having been listed
in the device tree to use grants.

Add a new config item to always force use of grants for virtio.

Fixes: fa1f57421e0b ("xen/virtio: Enable restricted memory access using Xen grant mappings")
Reported-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Tested-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com> # Arm64 guest using Xen
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Link: https://lore.kernel.org/r/20220622063838.8854-4-jgross@suse.com
Signed-off-by: Juergen Gross <jgross@suse.com>

xen/virtio: Enable restricted memory access using Xen grant mappings

In order to support virtio in Xen guests add a config option XEN_VIRTIO
enabling the user to specify whether in all Xen guests virtio should
be able to access memory via Xen grant mappings only on the host side.

Also set PLATFORM_VIRTIO_RESTRICTED_MEM_ACCESS feature from the guest
initialization code on Arm and x86 if CONFIG_XEN_VIRTIO is enabled.

Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Link: https://lore.kernel.org/r/1654197833-25362-5-git-send-email-olekstysh@gmail.com
Signed-off-by: Juergen Gross <jgross@suse.com>

arm/xen: Fix some refcount leaks

The of_find_compatible_node() function returns a node pointer with
refcount incremented, We should use of_node_put() on it when done
Add the missing of_node_put() to release the refcount.

Fixes: 9b08aaa3199a ("ARM: XEN: Move xen_early_init() before efi_init()")
Fixes: b2371587fe0c ("arm/xen: Read extended regions from DT and init Xen resource")
Signed-off-by: Miaoqian Lin <linmq006@gmail.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>

arm/xen: Read extended regions from DT and init Xen resource

This patch implements arch_xen_unpopulated_init() on Arm where
the extended regions (if any) are gathered from DT and inserted
into specific Xen resource to be used as unused address space
for Xen scratch pages by unpopulated-alloc code.

The extended region (safe range) is a region of guest physical
address space which is unused and could be safely used to create
grant/foreign mappings instead of wasting real RAM pages from
the domain memory for establishing these mappings.

The extended regions are chosen by the hypervisor at the domain
creation time and advertised to it via "reg" property under
hypervisor node in the guest device-tree. As region 0 is reserved
for grant table space (always present), the indexes for extended
regions are 1...N.

If arch_xen_unpopulated_init() fails for some reason the default
behaviour will be restored (allocate xenballooned pages).

This patch also removes XEN_UNPOPULATED_ALLOC dependency on x86.

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Link: https://lore.kernel.org/r/1639080336-26573-6-git-send-email-olekstysh@gmail.com
Signed-off-by: Juergen Gross <jgross@suse.com>

arm/xen: Switch to use gnttab_setup_auto_xlat_frames() for DT

Read the start address of the grant table space from DT
(region 0).

This patch mostly restores behaviour before commit 3cf4095d7446
("arm/xen: Use xen_xlate_map_ballooned_pages to setup grant table")
but trying not to break the ACPI support added after that commit.
So the patch touches DT part only and leaves the ACPI part with
xen_xlate_map_ballooned_pages(). Also in order to make a code more
resilient use a fallback to xen_xlate_map_ballooned_pages() if grant
table region wasn't found.

This is a preparation for using Xen extended region feature
where unused regions of guest physical address space (provided
by the hypervisor) will be used to create grant/foreign/whatever
mappings instead of wasting real RAM pages from the domain memory
for establishing these mappings.

The immediate benefit of this change:
- Avoid superpage shattering in Xen P2M when establishing
stage-2 mapping (GFN <-> MFN) for the grant table space
- Avoid wasting real RAM pages (reducing the amount of memory
usuable) for mapping grant table space
- The grant table space is always mapped at the exact
same place (region 0 is reserved for the grant table)

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Link: https://lore.kernel.org/r/1639080336-26573-3-git-send-email-olekstysh@gmail.com
Signed-off-by: Juergen Gross <jgross@suse.com>

xen: allow pv-only hypercalls only with CONFIG_XEN_PV

Put the definitions of the hypercalls usable only by pv guests inside
CONFIG_XEN_PV sections.

On Arm two dummy functions related to pv hypercalls can be removed.

While at it remove the no longer supported tmem hypercall definition.

Signed-off-by: Juergen Gross <jgross@suse.com>
Link: https://lore.kernel.org/r/20211028081221.2475-3-jgross@suse.com
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

ARM: 9092/1: xen: Register with kernel restart handler

Register with kernel restart handler instead of setting arm_pm_restart
directly.

Select a high priority of 192 to ensure that default restart handlers
are replaced if Xen is running.

Acked-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Wolfram Sang <wsa+renesas@sang-engineering.com>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>

arm/xen: Don't probe xenbus as part of an early initcall

After Commit 3499ba8198cad ("xen: Fix event channel callback via
INTX/GSI"), xenbus_probe() will be called too early on Arm. This will
recent to a guest hang during boot.

If the hang wasn't there, we would have ended up to call
xenbus_probe() twice (the second time is in xenbus_probe_initcall()).

We don't need to initialize xenbus_probe() early for Arm guest.
Therefore, the call in xen_guest_init() is now removed.

After this change, there is no more external caller for xenbus_probe().
So the function is turned to a static one. Interestingly there were two
prototypes for it.

Cc: stable@vger.kernel.org
Fixes: 3499ba8198cad ("xen: Fix event channel callback via INTX/GSI")
Reported-by: Ian Jackson <iwj@xenproject.org>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Link: https://lore.kernel.org/r/20210210170654.5377-1-julien@xen.org
Signed-off-by: Juergen Gross <jgross@suse.com>

xen: Fix event channel callback via INTX/GSI

For a while, event channel notification via the PCI platform device
has been broken, because we attempt to communicate with xenstore before
we even have notifications working, with the xs_reset_watches() call
in xs_init().

We tend to get away with this on Xen versions below 4.0 because we avoid
calling xs_reset_watches() anyway, because xenstore might not cope with
reading a non-existent key. And newer Xen *does* have the vector
callback support, so we rarely fall back to INTX/GSI delivery.

To fix it, clean up a bit of the mess of xs_init() and xenbus_probe()
startup. Call xs_init() directly from xenbus_init() only in the !XS_HVM
case, deferring it to be called from xenbus_probe() in the XS_HVM case
instead.

Then fix up the invocation of xenbus_probe() to happen either from its
device_initcall if the callback is available early enough, or when the
callback is finally set up. This means that the hack of calling
xenbus_probe() from a workqueue after the first interrupt, or directly
from the PCI platform device setup, is no longer needed.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Link: https://lore.kernel.org/r/20210113132606.422794-2-dwmw2@infradead.org
Signed-off-by: Juergen Gross <jgross@suse.com>

arm/arm64: xen: Fix to convert percpu address to gfn correctly

Use per_cpu_ptr_to_phys() instead of virt_to_phys() for per-cpu
address conversion.

In xen_starting_cpu(), per-cpu xen_vcpu_info address is converted
to gfn by virt_to_gfn() macro. However, since the virt_to_gfn(v)
assumes the given virtual address is in linear mapped kernel memory
area, it can not convert the per-cpu memory if it is allocated on
vmalloc area.

This depends on CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK.
If it is enabled, the first chunk of percpu memory is linear mapped.
In the other case, that is allocated from vmalloc area. Moreover,
if the first chunk of percpu has run out until allocating
xen_vcpu_info, it will be allocated on the 2nd chunk, which is
based on kernel memory or vmalloc memory (depends on
CONFIG_NEED_PER_CPU_KM).

Without this fix and kernel configured to use vmalloc area for
the percpu memory, the Dom0 kernel will fail to boot with following
errors.

[ 0.466172] Xen: initializing cpu0
[ 0.469601] ------------[ cut here ]------------
[ 0.474295] WARNING: CPU: 0 PID: 1 at arch/arm64/xen/../../arm/xen/enlighten.c:153 xen_starting_cpu+0x160/0x180
[ 0.484435] Modules linked in:
[ 0.487565] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.9.0-rc4+ #4
[ 0.493895] Hardware name: Socionext Developer Box (DT)
[ 0.499194] pstate: 00000005 (nzcv daif -PAN -UAO BTYPE=--)
[ 0.504836] pc : xen_starting_cpu+0x160/0x180
[ 0.509263] lr : xen_starting_cpu+0xb0/0x180
[ 0.513599] sp : ffff8000116cbb60
[ 0.516984] x29: ffff8000116cbb60 x28: ffff80000abec000
[ 0.522366] x27: 0000000000000000 x26: 0000000000000000
[ 0.527754] x25: ffff80001156c000 x24: fffffdffbfcdb600
[ 0.533129] x23: 0000000000000000 x22: 0000000000000000
[ 0.538511] x21: ffff8000113a99c8 x20: ffff800010fe4f68
[ 0.543892] x19: ffff8000113a9988 x18: 0000000000000010
[ 0.549274] x17: 0000000094fe0f81 x16: 00000000deadbeef
[ 0.554655] x15: ffffffffffffffff x14: 0720072007200720
[ 0.560037] x13: 0720072007200720 x12: 0720072007200720
[ 0.565418] x11: 0720072007200720 x10: 0720072007200720
[ 0.570801] x9 : ffff8000100fbdc0 x8 : ffff800010715208
[ 0.576182] x7 : 0000000000000054 x6 : ffff00001b790f00
[ 0.581564] x5 : ffff800010bbf880 x4 : 0000000000000000
[ 0.586945] x3 : 0000000000000000 x2 : ffff80000abec000
[ 0.592327] x1 : 000000000000002f x0 : 0000800000000000
[ 0.597716] Call trace:
[ 0.600232] xen_starting_cpu+0x160/0x180
[ 0.604309] cpuhp_invoke_callback+0xac/0x640
[ 0.608736] cpuhp_issue_call+0xf4/0x150
[ 0.612728] __cpuhp_setup_state_cpuslocked+0x128/0x2c8
[ 0.618030] __cpuhp_setup_state+0x84/0xf8
[ 0.622192] xen_guest_init+0x324/0x364
[ 0.626097] do_one_initcall+0x54/0x250
[ 0.630003] kernel_init_freeable+0x12c/0x2c8
[ 0.634428] kernel_init+0x1c/0x128
[ 0.637988] ret_from_fork+0x10/0x18
[ 0.641635] ---[ end trace d95b5309a33f8b27 ]---
[ 0.646337] ------------[ cut here ]------------
[ 0.651005] kernel BUG at arch/arm64/xen/../../arm/xen/enlighten.c:158!
[ 0.657697] Internal error: Oops - BUG: 0 [#1] SMP
[ 0.662548] Modules linked in:
[ 0.665676] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W 5.9.0-rc4+ #4
[ 0.673398] Hardware name: Socionext Developer Box (DT)
[ 0.678695] pstate: 00000005 (nzcv daif -PAN -UAO BTYPE=--)
[ 0.684338] pc : xen_starting_cpu+0x178/0x180
[ 0.688765] lr : xen_starting_cpu+0x144/0x180
[ 0.693188] sp : ffff8000116cbb60
[ 0.696573] x29: ffff8000116cbb60 x28: ffff80000abec000
[ 0.701955] x27: 0000000000000000 x26: 0000000000000000
[ 0.707344] x25: ffff80001156c000 x24: fffffdffbfcdb600
[ 0.712718] x23: 0000000000000000 x22: 0000000000000000
[ 0.718107] x21: ffff8000113a99c8 x20: ffff800010fe4f68
[ 0.723481] x19: ffff8000113a9988 x18: 0000000000000010
[ 0.728863] x17: 0000000094fe0f81 x16: 00000000deadbeef
[ 0.734245] x15: ffffffffffffffff x14: 0720072007200720
[ 0.739626] x13: 0720072007200720 x12: 0720072007200720
[ 0.745008] x11: 0720072007200720 x10: 0720072007200720
[ 0.750390] x9 : ffff8000100fbdc0 x8 : ffff800010715208
[ 0.755771] x7 : 0000000000000054 x6 : ffff00001b790f00
[ 0.761153] x5 : ffff800010bbf880 x4 : 0000000000000000
[ 0.766534] x3 : 0000000000000000 x2 : 00000000deadbeef
[ 0.771916] x1 : 00000000deadbeef x0 : ffffffffffffffea
[ 0.777304] Call trace:
[ 0.779819] xen_starting_cpu+0x178/0x180
[ 0.783898] cpuhp_invoke_callback+0xac/0x640
[ 0.788325] cpuhp_issue_call+0xf4/0x150
[ 0.792317] __cpuhp_setup_state_cpuslocked+0x128/0x2c8
[ 0.797619] __cpuhp_setup_state+0x84/0xf8
[ 0.801779] xen_guest_init+0x324/0x364
[ 0.805683] do_one_initcall+0x54/0x250
[ 0.809590] kernel_init_freeable+0x12c/0x2c8
[ 0.814016] kernel_init+0x1c/0x128
[ 0.817583] ret_from_fork+0x10/0x18
[ 0.821226] Code: d0006980 f9427c00 cb000300 17ffffea (d4210000)
[ 0.827415] ---[ end trace d95b5309a33f8b28 ]---
[ 0.832076] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[ 0.839815] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]---

Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Link: https://lore.kernel.org/r/160196697165.60224.17470743378683334995.stgit@devnote2
Signed-off-by: Juergen Gross <jgross@suse.com>

xen/arm: do not setup the runstate info page if kpti is enabled

The VCPUOP_register_runstate_memory_area hypercall takes a virtual
address of a buffer as a parameter. The semantics of the hypercall are
such that the virtual address should always be valid.

When KPTI is enabled and we are running userspace code, the virtual
address is not valid, thus, Linux is violating the semantics of
VCPUOP_register_runstate_memory_area.

Do not call VCPUOP_register_runstate_memory_area when KPTI is enabled.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
CC: Bertrand Marquis <Bertrand.Marquis@arm.com>
CC: boris.ostrovsky@oracle.com
CC: jgross@suse.com
Link: https://lore.kernel.org/r/20200924234955.15455-1-sstabellini@kernel.org
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

arm/xen: remove the unused macro GRANT_TABLE_PHYSADDR

Fix the following sparse warning:

arch/arm64/xen/../../arm/xen/enlighten.c:244: warning: macro
"GRANT_TABLE_PHYSADDR" is not used [-Wunused-macros]

It is an isolated macro, and should be removed when its last user
was deleted in the following commit 3cf4095d7446 ("arm/xen: Use
xen_xlate_map_ballooned_pages to setup grant table")

Signed-off-by: Xiaofei Tan <tanxiaofei@huawei.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>

arm/xen: make _xen_start_info static

Fix the following sparse warning:

arch/arm64/xen/../../arm/xen/enlighten.c:39:19: warning: symbol
'_xen_start_info' was not declared. Should it be static?

Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Jason Yan <yanaijie@huawei.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Link: https://lore.kernel.org/r/20200415084853.5808-1-yanaijie@huawei.com
Signed-off-by: Juergen Gross <jgross@suse.com>

xen/efi: have a common runtime setup function

Today the EFI runtime functions are setup in architecture specific
code (x86 and arm), with the functions themselves living in drivers/xen
as they are not architecture dependent.

As the setup is exactly the same for arm and x86 move the setup to
drivers/xen, too. This at once removes the need to make the single
functions global visible.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
[boris: "Dropped EXPORT_SYMBOL_GPL(xen_efi_runtime_setup)"]
Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

ARM: xen: unexport HYPERVISOR_platform_op function

HYPERVISOR_platform_op() is an inline function and should not
be exported. Since commit 15bfc2348d54 ("modpost: check for
static EXPORT_SYMBOL* functions"), this causes a warning:

WARNING: "HYPERVISOR_platform_op" [vmlinux] is a static EXPORT_SYMBOL_GPL

Instead, export the underlying function called by the static inline:
HYPERVISOR_platform_op_raw.

Fixes: 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL* functions")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>

treewide: Add SPDX license identifier for missed files

Add SPDX license identifiers to all files which:

- Have no license information of any form

- Have EXPORT_.*_SYMBOL_GPL inside which was used in the
initial scan/conversion to ignore the file

These files fall under the project license, GPL v2 only. The resulting SPDX
license identifier is:

GPL-2.0-only

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

x86/xen: Move pv specific parts of arch/x86/xen/mmu.c to mmu_pv.c

There are some PV specific functions in arch/x86/xen/mmu.c which can be
moved to mmu_pv.c. This in turn enables to build multicalls.c dependent
on CONFIG_XEN_PV.

Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: xen-devel@lists.xenproject.org
Cc: virtualization@lists.linux-foundation.org
Cc: akataria@vmware.com
Cc: rusty@rustcorp.com.au
Cc: hpa@zytor.com
Link: https://lkml.kernel.org/r/20180828074026.820-3-jgross@suse.com

xen: share start flags between PV and PVH

Use a global variable to store the start flags for both PV and PVH.
This allows the xen_initial_domain macro to work properly on PVH.

Note that ARM is also switched to use the new variable.

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Juergen Gross <jgross@suse.com>

xen/privcmd: add IOCTL_PRIVCMD_MMAP_RESOURCE

My recent Xen patch series introduces a new HYPERVISOR_memory_op to
support direct priv-mapping of certain guest resources (such as ioreq
pages, used by emulators) by a tools domain, rather than having to access
such resources via the guest P2M.

This patch adds the necessary infrastructure to the privcmd driver and
Xen MMU code to support direct resource mapping.

NOTE: The adjustment in the MMU code is partially cosmetic. Xen will now
allow a PV tools domain to map guest pages either by GFN or MFN, thus
the term 'mfn' has been swapped for 'pfn' in the lower layers of the
remap code.

Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Signed-off-by: Juergen Gross <jgross@suse.com>

arm/xen: Consolidate calls to shutdown hypercall in a single helper

Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Juergen Gross <jgross@suse.com>

xen/privcmd: Add IOCTL_PRIVCMD_DM_OP

Recently a new dm_op[1] hypercall was added to Xen to provide a mechanism
for restricting device emulators (such as QEMU) to a limited set of
hypervisor operations, and being able to audit those operations in the
kernel of the domain in which they run.

This patch adds IOCTL_PRIVCMD_DM_OP as gateway for __HYPERVISOR_dm_op.

NOTE: There is no requirement for user-space code to bounce data through
locked memory buffers (as with IOCTL_PRIVCMD_HYPERCALL) since
privcmd has enough information to lock the original buffers
directly.

[1] http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=524a98c2

Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

cpu/hotplug: Cleanup state names

When the state names got added a script was used to add the extra argument
to the calls. The script basically converted the state constant to a
string, but the cleanup to convert these strings into meaningful ones did
not happen.

Replace all the useless strings with 'subsys/xxx/yyy:state' strings which
are used in all the other places already.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Link: http://lkml.kernel.org/r/20161221192112.085444152@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

arm/xen: Use alloc_percpu rather than __alloc_percpu

The function xen_guest_init is using __alloc_percpu with an alignment
which are not power of two.

However, the percpu allocator never supported alignments which are not power
of two and has always behaved incorectly in thise case.

Commit 3ca45a4 "percpu: ensure requested alignment is power of two"
introduced a check which trigger a warning [1] when booting linux-next
on Xen. But in reality this bug was always present.

This can be fixed by replacing the call to __alloc_percpu with
alloc_percpu. The latter will use an alignment which are a power of two.

[1]

[ 0.023921] illegal size (48) or align (48) for percpu allocation
[ 0.024167] ------------[ cut here ]------------
[ 0.024344] WARNING: CPU: 0 PID: 1 at linux/mm/percpu.c:892 pcpu_alloc+0x88/0x6c0
[ 0.024584] Modules linked in:
[ 0.024708]
[ 0.024804] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
4.9.0-rc7-next-20161128 #473
[ 0.025012] Hardware name: Foundation-v8A (DT)
[ 0.025162] task: ffff80003d870000 task.stack: ffff80003d844000
[ 0.025351] PC is at pcpu_alloc+0x88/0x6c0
[ 0.025490] LR is at pcpu_alloc+0x88/0x6c0
[ 0.025624] pc : [<ffff00000818e678>] lr : [<ffff00000818e678>]
pstate: 60000045
[ 0.025830] sp : ffff80003d847cd0
[ 0.025946] x29: ffff80003d847cd0 x28: 0000000000000000
[ 0.026147] x27: 0000000000000000 x26: 0000000000000000
[ 0.026348] x25: 0000000000000000 x24: 0000000000000000
[ 0.026549] x23: 0000000000000000 x22: 00000000024000c0
[ 0.026752] x21: ffff000008e97000 x20: 0000000000000000
[ 0.026953] x19: 0000000000000030 x18: 0000000000000010
[ 0.027155] x17: 0000000000000a3f x16: 00000000deadbeef
[ 0.027357] x15: 0000000000000006 x14: ffff000088f79c3f
[ 0.027573] x13: ffff000008f79c4d x12: 0000000000000041
[ 0.027782] x11: 0000000000000006 x10: 0000000000000042
[ 0.027995] x9 : ffff80003d847a40 x8 : 6f697461636f6c6c
[ 0.028208] x7 : 6120757063726570 x6 : ffff000008f79c84
[ 0.028419] x5 : 0000000000000005 x4 : 0000000000000000
[ 0.028628] x3 : 0000000000000000 x2 : 000000000000017f
[ 0.028840] x1 : ffff80003d870000 x0 : 0000000000000035
[ 0.029056]
[ 0.029152] ---[ end trace 0000000000000000 ]---
[ 0.029297] Call trace:
[ 0.029403] Exception stack(0xffff80003d847b00 to
0xffff80003d847c30)
[ 0.029621] 7b00: 0000000000000030 0001000000000000
ffff80003d847cd0 ffff00000818e678
[ 0.029901] 7b20: 0000000000000002 0000000000000004
ffff000008f7c060 0000000000000035
[ 0.030153] 7b40: ffff000008f79000 ffff000008c4cd88
ffff80003d847bf0 ffff000008101778
[ 0.030402] 7b60: 0000000000000030 0000000000000000
ffff000008e97000 00000000024000c0
[ 0.030647] 7b80: 0000000000000000 0000000000000000
0000000000000000 0000000000000000
[ 0.030895] 7ba0: 0000000000000035 ffff80003d870000
000000000000017f 0000000000000000
[ 0.031144] 7bc0: 0000000000000000 0000000000000005
ffff000008f79c84 6120757063726570
[ 0.031394] 7be0: 6f697461636f6c6c ffff80003d847a40
0000000000000042 0000000000000006
[ 0.031643] 7c00: 0000000000000041 ffff000008f79c4d
ffff000088f79c3f 0000000000000006
[ 0.031877] 7c20: 00000000deadbeef 0000000000000a3f
[ 0.032051] [<ffff00000818e678>] pcpu_alloc+0x88/0x6c0
[ 0.032229] [<ffff00000818ece8>] __alloc_percpu+0x18/0x20
[ 0.032409] [<ffff000008d9606c>] xen_guest_init+0x174/0x2f4
[ 0.032591] [<ffff0000080830f8>] do_one_initcall+0x38/0x130
[ 0.032783] [<ffff000008d90c34>] kernel_init_freeable+0xe0/0x248
[ 0.032995] [<ffff00000899a890>] kernel_init+0x10/0x100
[ 0.033172] [<ffff000008082ec0>] ret_from_fork+0x10/0x50

Reported-by: Wei Chen <wei.chen@arm.com>
Link: https://lkml.org/lkml/2016/11/28/669
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Cc: stable@vger.kernel.org

arm/xen: fix SMP guests boot

Commit 88e957d6e47f ("xen: introduce xen_vcpu_id mapping") broke SMP
ARM guests on Xen. When FIFO-based event channels are in use (this is
the default), evtchn_fifo_alloc_control_block() is called on
CPU_UP_PREPARE event and this happens before we set up xen_vcpu_id
mapping in xen_starting_cpu. Temporary fix the issue by setting direct
Linux CPU id <-> Xen vCPU id mapping for all possible CPUs at boot. We
don't currently support kexec/kdump on Xen/ARM so these ids always
match.

In future, we have several ways to solve the issue, e.g.:

- Eliminate all hypercalls from CPU_UP_PREPARE, do them from the
starting CPU. This can probably be done for both x86 and ARM and, if
done, will allow us to get Xen's idea of vCPU id from CPUID/MPIDR on
the starting CPU directly, no messing with ACPI/device tree
required.

- Save vCPU id information from ACPI/device tree on ARM and use it to
initialize xen_vcpu_id mapping. This is the same trick we currently
do on x86.

Reported-by: Julien Grall <julien.grall@arm.com>
Tested-by: Wei Chen <Wei.Chen@arm.com>
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>

xen: change the type of xen_vcpu_id to uint32_t

We pass xen_vcpu_id mapping information to hypercalls which require
uint32_t type so it would be cleaner to have it as uint32_t. The
initializer to -1 can be dropped as we always do the mapping before using
it and we never check the 'not set' value anyway.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>

x86/xen: use xen_vcpu_id mapping for HYPERVISOR_vcpu_op

HYPERVISOR_vcpu_op() passes Linux's idea of vCPU id as a parameter
while Xen's idea is expected. In some cases these ideas diverge so we
need to do remapping.

Convert all callers of HYPERVISOR_vcpu_op() to use xen_vcpu_nr().

Leave xen_fill_possible_map() and xen_filter_cpu_maps() intact as
they're only being called by PV guests before perpu areas are
initialized. While the issue could be solved by switching to
early_percpu for xen_vcpu_id I think it's not worth it: PV guests will
probably never get to the point where their idea of vCPU id diverges
from Xen's.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>

xen: introduce xen_vcpu_id mapping

It may happen that Xen's and Linux's ideas of vCPU id diverge. In
particular, when we crash on a secondary vCPU we may want to do kdump
and unlike plain kexec where we do migrate_to_reboot_cpu() we try
booting on the vCPU which crashed. This doesn't work very well for
PVHVM guests as we have a number of hypercalls where we pass vCPU id
as a parameter. These hypercalls either fail or do something
unexpected.

To solve the issue introduce percpu xen_vcpu_id mapping. ARM and PV
guests get direct mapping for now. Boot CPU for PVHVM guest gets its
id from CPUID. With secondary CPUs it is a bit more
trickier. Currently, we initialize IPI vectors before these CPUs boot
so we can't use CPUID. Use ACPI ids from MADT instead.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>

arm/xen: Convert to hotplug state machine

Install the callbacks via the state machine and let the core invoke
the callbacks on the already online CPUs.

The get_cpu() in xen_starting_cpu() boils down to preempt_disable() since
we already know the CPU we run on. Disabling preemption shouldn't be required
here from what I see since it we don't switch CPUs while invoking the function.

Signed-off-by: Richard Cochran <rcochran@linutronix.de>
Signed-off-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arm-kernel@lists.infradead.org
Cc: rt@linutronix.de
Cc: xen-devel@lists.xenproject.org
Link: http://lkml.kernel.org/r/20160713153336.971559670@linutronix.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>

arm/xen: add support for vm_assist hypercall

Add support for the Xen HYPERVISOR_vm_assist hypercall.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Reviewed-by: Julien Grall <julien.grall@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>

xen: add steal_clock support on x86

The pv_time_ops structure contains a function pointer for the
"steal_clock" functionality used only by KVM and Xen on ARM. Xen on x86
uses its own mechanism to account for the "stolen" time a thread wasn't
able to run due to hypervisor scheduling.

Add support in Xen arch independent time handling for this feature by
moving it out of the arm arch into drivers/xen and remove the x86 Xen
hack.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>

Xen: EFI: Parse DT parameters for Xen specific UEFI

The EFI DT parameters for bare metal are located under /chosen node,
while for Xen Dom0 they are located under /hyperviosr/uefi node. These
parameters under /chosen and /hyperviosr/uefi are not expected to appear
at the same time.

Parse these EFI parameters and initialize EFI like the way for bare
metal except the runtime services because the runtime services for Xen
Dom0 are available through hypercalls and they are always enabled. So it
sets the EFI_RUNTIME_SERVICES flag if it finds /hyperviosr/uefi node and
bails out in arm_enable_runtime_services() when EFI_RUNTIME_SERVICES
flag is set already.

Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Signed-off-by: Matt Fleming <matt@codeblueprint.co.uk>

ARM: XEN: Move xen_early_init() before efi_init()

Move xen_early_init() before efi_init(), then when calling efi_init()
could initialize Xen specific UEFI.

Check if it runs on Xen hypervisor through the flat dts.

Cc: Russell King <linux@arm.linux.org.uk>
Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Reviewed-by: Julien Grall <julien.grall@arm.com>
Tested-by: Julien Grall <julien.grall@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>

arm/xen: Get event-channel irq through HVM_PARAM when booting with ACPI

The kernel will get the event-channel IRQ through
HVM_PARAM_CALLBACK_IRQ.

Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Reviewed-by: Julien Grall <julien.grall@arm.com>
Tested-by: Julien Grall <julien.grall@arm.com>

arm/xen: Use xen_xlate_map_ballooned_pages to setup grant table

Use xen_xlate_map_ballooned_pages to setup grant table. Then it doesn't
rely on DT or ACPI to pass the start address and size of grant table.

Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Reviewed-by: Julien Grall <julien.grall@arm.com>
Tested-by: Julien Grall <julien.grall@arm.com>

xen/arm: set the system time in Xen via the XENPF_settime64 hypercall

If Linux is running as dom0, call XENPF_settime64 to update the system
time in Xen on pvclock_gtod notifications.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>

xen/arm: introduce xen_read_wallclock

Read the wallclock from the shared info page at boot time.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>

xen/arm: introduce HYPERVISOR_platform_op on arm and arm64

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/arm: account for stolen ticks

Register the runstate_memory_area with the hypervisor.
Use pv_time_ops.steal_clock to account for stolen ticks.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/arm: don't try to re-register vcpu_info on cpu_hotplug.

Call disable_percpu_irq on CPU_DYING and enable_percpu_irq when the cpu
is coming up.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Reviewed-by: Julien Grall <julien.grall@citrix.com>

xen/arm: correct comment in enlighten.c

Correct a comment in arch/arm/xen/enlighten.c referencing a wrong
source file.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>

arm/xen: Add support for 64KB page granularity

The hypercall interface is always using 4KB page granularity. This is
requiring to use xen page definition macro when we deal with hypercall.

Note that pfn_to_gfn is working with a Xen pfn (i.e 4KB). We may want to
rename pfn_gfn to make this explicit.

We also allocate a 64KB page for the shared page even though only the
first 4KB is used. I don't think this is really important for now as it
helps to have the pointer 4KB aligned (XENMEM_add_to_physmap is taking a
Xen PFN).

Signed-off-by: Julien Grall <julien.grall@citrix.com>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>

xen/privcmd: Further s/MFN/GFN/ clean-up

The privcmd code is mixing the usage of GFN and MFN within the same
functions which make the code difficult to understand when you only work
with auto-translated guests.

The privcmd driver is only dealing with GFN so replace all the mention
of MFN into GFN.

The ioctl structure used to map foreign change has been left unchanged
given that the userspace is using it. Nonetheless, add a comment to
explain the expected value within the "mfn" field.

Signed-off-by: Julien Grall <julien.grall@citrix.com>
Reviewed-by: David Vrabel <david.vrabel@citrix.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>

arm/xen: Drop the definition of xen_pci_platform_unplug

The commit 6f6c15ef912465b3aaafe709f39bd6026a8b3e72 "xen/pvhvm: Remove
the xen_platform_pci int." makes the x86 version of
xen_pci_platform_unplug static.

Therefore we don't need anymore to define a dummy xen_pci_platform_unplug
for ARM.

Signed-off-by: Julien Grall <julien.grall@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/events: Support event channel rebind on ARM

Currently, the event channel rebind code is gated with the presence of
the vector callback.

The virtual interrupt controller on ARM has the concept of per-CPU
interrupt (PPI) which allow us to support per-VCPU event channel.
Therefore there is no need of vector callback for ARM.

Xen is already using a free PPI to notify the guest VCPU of an event.
Furthermore, the xen code initialization in Linux (see
arch/arm/xen/enlighten.c) is requesting correctly a per-CPU IRQ.

Introduce new helper xen_support_evtchn_rebind to allow architecture
decide whether rebind an event is support or not. It will always return
true on ARM and keep the same behavior on x86.

This is also allow us to drop the usage of xen_have_vector_callback
entirely in the ARM code.

Signed-off-by: Julien Grall <julien.grall@citrix.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>

xen/arm: allow console=hvc0 to be omitted for guests

From: Ard Biesheuvel <ard.biesheuvel@linaro.org>

This patch registers hvc0 as the preferred console if no console
has been specified explicitly on the kernel command line.

The purpose is to allow platform agnostic kernels and boot images
(such as distro installers) to boot in a Xen/ARM domU without the
need to modify the command line by hand.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Reviewed-by: Julien Grall <julien.grall@linaro.org>

arm,arm64/xen: move Xen initialization earlier

Currently, Xen is initialized/discovered in an initcall. This doesn't
allow us to support earlyprintk or choosing the preferred console when
running on Xen.

The current function xen_guest_init is now split in 2 parts:
- xen_early_init: Check if there is a Xen node in the device tree
and setup domain type
- xen_guest_init: Retrieve the information from the device node and
initialize Xen (grant table, shared page...)

The former is called in setup_arch, while the latter is an initcall.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Will Deacon <will.deacon@arm.com>

arm/xen: Correctly check if the event channel interrupt is present

The function irq_of_parse_and_map returns 0 when the IRQ is not found.

Futhermore, move the check before notifying the user that we are running on
Xen.

Signed-off-by: Julien Grall <julien.grall@linaro.org>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>

xen/arm: Define xen_arch_suspend()

Commit 2b953a5e994c ("xen: Suspend ticks on all CPUs during suspend")
introduced xen_arch_suspend() routine but did so only for x86, breaking
ARM builds.

We need to add it to ARM as well.

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Reported-by: Michal Suchanek <hramrach@gmail.com>
Tested-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>

xen/privcmd: improve performance of MMAPBATCH_V2

Make the IOCTL_PRIVCMD_MMAPBATCH_V2 (and older V1 version) map
multiple frames at a time rather than one at a time, despite the pages
being non-consecutive GFNs.

xen_remap_foreign_mfn_array() is added which maps an array of GFNs
(instead of a consecutive range of GFNs).

Since per-frame errors are returned in an array, privcmd must set the
MMAPBATCH_V1 error bits as part of the "report errors" phase, after
all the frames are mapped.

Migrate times are significantly improved (when using a PV toolstack
domain). For example, for an idle 12 GiB PV guest:

Before After
real 0m38.179s 0m26.868s
user 0m15.096s 0m13.652s
sys 0m28.988s 0m18.732s

Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen: unify foreign GFN map/unmap for auto-xlated physmap guests

Auto-translated physmap guests (arm, arm64 and x86 PVHVM/PVH) map and
unmap foreign GFNs using the same method (updating the physmap).
Unify the two arm and x86 implementations into one commont one.

Note that on arm and arm64, the correct error code will be returned
(instead of always -EFAULT) and map/unmap failure warnings are no
longer printed. These changes are required if the foreign domain is
paging (-ENOENT failures are expected and must be propagated up to the
caller).

Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

arm64: Relax licensing of arm64 Xen DMA operations

With Xen configured into the arm64 kernel, any driver allocating
DMA'able memory for PCI operations, must be GPL compatible, regardless
of its interaction with Xen. This patch relaxes the GPL requirement of
xen_dma_ops and its dependencies to allow open source drivers to be
compiled for the arm64 architecture.

Signed-off-by: Chuck Tuffli <chuck.tuffli@emulex.com>

xen/arm: remove handling of XENFEAT_grant_map_identity

The feature has been removed from Xen. Also Linux cannot use it on ARM32
without CONFIG_ARM_LPAE.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>

xen/arm: introduce XENFEAT_grant_map_identity

The flag tells us that the hypervisor maps a grant page to guest
physical address == machine address of the page in addition to the
normal grant mapping address. It is needed to properly issue cache
maintenance operation at the completion of a DMA operation involving a
foreign grant.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Tested-by: Denis Schneider <v1ne2go@gmail.com>

xen/arm: use BUG_ON

Use BUG_ON(x) rather than if(x) BUG();

The semantic patch that fixes this problem is as follows:

// <smpl>
@@ identifier x; @@
-if (x) BUG();
+BUG_ON(x);
// </smpl>

Signed-off-by: Himangi Saraogi <himangi774@gmail.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

arm,arm64/xen: introduce HYPERVISOR_suspend()

Introduce HYPERVISOR_suspend() and a few additional empty stubs for
Xen arch specific functions called by drivers/xen/manage.c.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>

arm: xen: export HYPERVISOR_multicall to modules.

"arm: xen: implement multicall hypercall support." forgot to do this.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/gnttab: Use phys_addr_t to describe the grant frame base address

On ARM, address size can be 32 bits or 64 bits (if CONFIG_ARCH_PHYS_ADDR_T_64BIT
is enabled).
We can't assume that the grant frame base address will always fits in an
unsigned long. Use phys_addr_t instead of unsigned long as argument for
gnttab_setup_auto_xlat_frames.

Signed-off-by: Julien Grall <julien.grall@linaro.org>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: David Vrabel <david.vrabel@citrix.com>

arm/xen: Initialize event channels earlier

Event channels driver needs to be initialized very early. Until now, Xen
initialization was done after all CPUs was bring up.

We can safely move the initialization to an early initcall.

Also use a cpu notifier to:
- Register the VCPU when the CPU is prepared
- Enable event channel IRQ when the CPU is running

Signed-off-by: Julien Grall <julien.grall@linaro.org>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/grant: Implement an grant frame array struct (v3).

The 'xen_hvm_resume_frames' used to be an 'unsigned long'
and contain the virtual address of the grants. That was OK
for most architectures (PVHVM, ARM) were the grants are contiguous
in memory. That however is not the case for PVH - in which case
we will have to do a lookup for each virtual address for the PFN.

Instead of doing that, lets make it a structure which will contain
the array of PFNs, the virtual address and the count of said PFNs.

Also provide a generic functions: gnttab_setup_auto_xlat_frames and
gnttab_free_auto_xlat_frames to populate said structure with
appropriate values for PVHVM and ARM.

To round it off, change the name from 'xen_hvm_resume_frames' to
a more descriptive one - 'xen_auto_xlat_grant_frames'.

For PVH, in patch "xen/pvh: Piggyback on PVHVM for grant driver"
we will populate the 'xen_auto_xlat_grant_frames' by ourselves.

v2 moves the xen_remap in the gnttab_setup_auto_xlat_frames
and also introduces xen_unmap for gnttab_free_auto_xlat_frames.

Suggested-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
[v3: Based on top of 'asm/xen/page.h: remove redundant semicolon']
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

arm: xen: foreign mapping PTEs are special.

These mappings are in fact special and require special handling in privcmd,
which already exists. Failure to mark the PTE as special on arm64 causes all
sorts of bad PTE fun. e.g.

e.g.:

BUG: Bad page map in process xl pte:e0004077b33f53 pmd:4079575003
page:ffffffbce1a2f328 count:1 mapcount:-1 mapping: (null) index:0x0
page flags: 0x4000000000000014(referenced|dirty)
addr:0000007fb5259000 vm_flags:040644fa anon_vma: (null) mapping:ffffffc03a6fda58 index:0
vma->vm_ops->fault: privcmd_fault+0x0/0x38
vma->vm_file->f_op->mmap: privcmd_mmap+0x0/0x2c
CPU: 0 PID: 2657 Comm: xl Not tainted 3.12.0+ #102
Call trace:
[<ffffffc0000880f8>] dump_backtrace+0x0/0x12c
[<ffffffc000088238>] show_stack+0x14/0x1c
[<ffffffc0004b67e0>] dump_stack+0x70/0x90
[<ffffffc000125690>] print_bad_pte+0x12c/0x1bc
[<ffffffc0001268f4>] unmap_single_vma+0x4cc/0x700
[<ffffffc0001273b4>] unmap_vmas+0x68/0xb4
[<ffffffc00012c050>] unmap_region+0xcc/0x1d4
[<ffffffc00012df20>] do_munmap+0x218/0x314
[<ffffffc00012e060>] vm_munmap+0x44/0x64
[<ffffffc00012ed78>] SyS_munmap+0x24/0x34

Where unmap_single_vma contains inlined -> unmap_page_range -> zap_pud_range
-> zap_pmd_range -> zap_pte_range -> print_bad_pte.

Or:

BUG: Bad page state in process xl pfn:4077b4d
page:ffffffbce1a2f8d8 count:0 mapcount:-1 mapping: (null) index:0x0
page flags: 0x4000000000000014(referenced|dirty)
Modules linked in:
CPU: 0 PID: 2657 Comm: xl Tainted: G B 3.12.0+ #102
Call trace:
[<ffffffc0000880f8>] dump_backtrace+0x0/0x12c
[<ffffffc000088238>] show_stack+0x14/0x1c
[<ffffffc0004b67e0>] dump_stack+0x70/0x90
[<ffffffc00010f798>] bad_page+0xc4/0x110
[<ffffffc00010f8b4>] free_pages_prepare+0xd0/0xd8
[<ffffffc000110e94>] free_hot_cold_page+0x28/0x178
[<ffffffc000111460>] free_hot_cold_page_list+0x38/0x60
[<ffffffc000114cf0>] release_pages+0x190/0x1dc
[<ffffffc00012c0e0>] unmap_region+0x15c/0x1d4
[<ffffffc00012df20>] do_munmap+0x218/0x314
[<ffffffc00012e060>] vm_munmap+0x44/0x64
[<ffffffc00012ed78>] SyS_munmap+0x24/0x34

x86 already gets this correct. 32-bit arm gets away with this because there is
not PTE_SPECIAL bit in the PTE there and the vm_normal_page fallback path does
the right thing.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

XEN: Grant table address, xen_hvm_resume_frames, is a phys_addr not a pfn

From: Eric Trudeau <etrudeau@broadcom.com>

xen_hvm_resume_frames stores the physical address of the grant table.
englighten.c was incorrectly setting it as if it was a page frame number.
This caused the table to be mapped into the guest at an unexpected physical
address.

Additionally, a warning is improved to include the grant table address which
failed in xen_remap.

Signed-off-by: Eric Trudeau <etrudeau@broadcom.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/arm: disable cpuidle and cpufreq when linux is running as dom0

When linux is running as dom0, Xen doesn't show the physical cpu but a
virtual CPU.
On some ARM SOC (for instance the exynos 5250), linux registers callbacks
for cpuidle and cpufreq. When these callbacks are called, they will modify
directly the physical cpu not the virtual one. It can impact the whole board
instead of only dom0.

Signed-off-by: Julien Grall <julien.grall@linaro.org>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

ARM: xen: only set pm function ptrs for Xen guests

xen_pm_init was unconditionally setting pm_power_off and arm_pm_restart
function pointers. This breaks multi-platform kernels. Make this
conditional on running as a Xen guest and make it a late_initcall to
ensure it is setup after platform code for Dom0.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: stable@vger.kernel.org

xen/arm: missing put_cpu in xen_percpu_init

When CONFIG_PREEMPT is enabled, Linux will not be able to boot and warn:
[ 4.127825] ------------[ cut here ]------------
[ 4.133376] WARNING: at init/main.c:699 do_one_initcall+0x150/0x158()
[ 4.140738] initcall xen_init_events+0x0/0x10c returned with preemption imbalance

This is because xen_percpu_init uses get_cpu but doesn't have the corresponding
put_cpu.

Signed-off-by: Julien Grall <julien.grall@linaro.org>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/arm,arm64: update xen_restart after ff701306cd49 and 7b6d864b48d9

Commit 7b6d864b48d9 (reboot: arm: change reboot_mode to use enum
reboot_mode) and ff701306cd49 (arm64: use common reboot infrastructure)
change the prototype of arm_pm_restart on arm and arm64.
Update xen_restart accordingly.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/arm and xen/arm64: implement HYPERVISOR_tmem_op

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/arm: rename xen_secondary_init and run it on every online cpu

Rename xen_secondary_init to xen_percpu_init.
Run xen_percpu_init on the each online cpu, reuse the current on_each_cpu call.
Merge xen_percpu_enable_events into xen_percpu_init.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/arm: do not handle VCPUOP_register_vcpu_info failures

We expect VCPUOP_register_vcpu_info to succeed, do not try to handle
failures.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>

xen/arm: initialize pm functions later

If we are running in dom0, we have to wait for the arch specific code to
complete the initialization in order for us to successfully reset the
power_off and pm_restart functions.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/arm: remove duplicated include from enlighten.c

Remove duplicated include.

Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>

xen/arm: use sched_op hypercalls for machine reboot and power off

Changes in v5:
- set pm_power_off and arm_pm_restart from the Xen specific
intialization code.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/arm: SMP support

Map vcpu_info using VCPUOP_register_vcpu_info on all the online vcpus,
make sure the allocated struct doesn't cross a page boundary.

Call enable_percpu_irq on every cpu.

Changes in v5:
- allocate xen_vcpu_info dynamically, aligning it to the size of the
struct;
- use VCPUOP_register_vcpu_info on cpu0 too.

Changed in v2:
- move the percpu variable argument fix to a separate patch;
- remove unused variable.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/arm: implement HYPERVISOR_vcpu_op

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Reviewed-by: Ian Campbell <ian.campbell@citrix.com>

xen/arm: actually pass a non-NULL percpu pointer to request_percpu_irq

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Reviewed-by: Ian Campbell <ian.camjpbell@citrix.com>
CC: stable@vger.kernel.org

xen: implement updated XENMEM_add_to_physmap_range ABI

Allows for more fine grained error reporting. Only used by PVH and
ARM both of which are marked EXPERIMENTAL precisely because the ABI
is not yet stable

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
[v1: Rebased without PVH patches]
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

xen: arm: implement remap interfaces needed for privcmd mappings.

We use XENMEM_add_to_physmap_range which is the preferred interface
for foreign mappings.

Acked-by: Mukesh Rathor <mukesh.rathor@oracle.com>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

xen: arm: enable balloon driver

The code is now in a state where can just enable it.

Drop the *_xenballloned_pages duplicates since these are now supplied
by the balloon code.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

xen/arm: Fix compile errors when drivers are compiled as modules (export more).

The commit 911dec0db4de6ccc544178a8ddaf9cec0a11d533
"xen/arm: Fix compile errors when drivers are compiled as modules." exports
the neccessary functions. But to guard ourselves against out-of-tree modules
and future drivers hitting this, lets export all of the relevant
hypercalls.

Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

xen/arm: Fix compile errors when drivers are compiled as modules.

We end up with:

ERROR: "HYPERVISOR_event_channel_op" [drivers/xen/xen-gntdev.ko] undefined!
ERROR: "privcmd_call" [drivers/xen/xen-privcmd.ko] undefined!
ERROR: "HYPERVISOR_grant_table_op" [drivers/net/xen-netback/xen-netback.ko] undefined!

and this patch exports said function (which is implemented in hypercall.S).

Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

xen/arm: implement alloc/free_xenballooned_pages with alloc_pages/kfree

Only until we get the balloon driver to work.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

xen/arm: receive Xen events on ARM

Compile events.c on ARM.
Parse, map and enable the IRQ to get event notifications from the device
tree (node "/xen").

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

xen/arm: initialize grant_table on ARM

Initialize the grant table mapping at the address specified at index 0
in the DT under the /xen node.
After the grant table is initialized, call xenbus_probe (if not dom0).

Changes in v2:

- introduce GRANT_TABLE_PHYSADDR;
- remove unneeded initialization of boot_max_nr_grant_frames.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

xen/arm: get privilege status

Use Xen features to figure out if we are privileged.

XENFEAT_dom0 was introduced by 23735 in xen-unstable.hg.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

xen/arm: Xen detection and shared_info page mapping

Check for a node in the device tree compatible with "xen,xen", if it is
present set xen_domain_type to XEN_HVM_DOMAIN and continue
initialization.

Map the real shared info page using XENMEM_add_to_physmap with
XENMAPSPACE_shared_info.

Changes in v4:

- simpler parsing of Xen version in the compatible DT node.

Changes in v3:

- use the "xen,xen" notation rather than "arm,xen";
- add an additional check on the presence of the Xen version.

Changes in v2:

- replace pr_info with pr_debug.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

arm: initial Xen support

- Basic hypervisor.h and interface.h definitions.
- Skeleton enlighten.c, set xen_start_info to an empty struct.
- Make xen_initial_domain dependent on the SIF_PRIVILIGED_BIT.

The new code only compiles when CONFIG_XEN is set, that is going to be
added to arch/arm/Kconfig in patch #11 "xen/arm: introduce CONFIG_XEN on
ARM".

Changes in v3:

- improve comments.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>