optee: ffa_abi: add asynchronous notifications

Adds support for asynchronous notifications from OP-TEE in secure world
when communicating via FF-A. In principle from OP-TEE and kernel driver
point of view this works in the same way as for the SMC ABI based
implementation.

The OP-TEE FF-A ABI is expanded in OPTEE_FFA_EXCHANGE_CAPABILITIES with
the capability OPTEE_FFA_SEC_CAP_ASYNC_NOTIF to indicate that OP-TEE
supports asynchronous notifications. OPTEE_FFA_ENABLE_ASYNC_NOTIF is
also added to tell that the driver has successfully initialized these
notifications.

Notification capability is negotiated while the driver is initialized.
If both sides supports these notifications then they are enabled.

The notification concept in this driver is merged with the FF-A concept,
the lower 64 values are reserved for FF-A as asynchronous notifications
while the synchronous notifications use the higher values.

So a FF-A notification has to be allocated for each discrete
asynchronous notification value needed. Only one asynchronous
notification value is used at the moment, the "do bottom half"
notification.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Tested-by: Sudeep Holla <sudeep.holla@arm.com>

tee: optee: support tracking system threads

Adds support in the OP-TEE driver to keep track of reserved system
threads. The logic allows one OP-TEE thread to be reserved to TEE system
sessions.

The optee_cq_*() functions are updated to handle this if enabled,
that is when TEE describes how many thread context it supports
and when at least 1 session has registered as a system session
(using tee_client_system_session()).

For sake of simplicity, initialization of call queue management
is factorized into new helper function optee_cq_init().

The SMC ABI part of the driver enables this tracking, but the
FF-A ABI part does not.

Co-developed-by: Jens Wiklander <jens.wiklander@linaro.org>
Co-developed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Etienne Carriere <etienne.carriere@foss.st.com>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: optee: system thread call property

Adds an argument to do_call_with_arg() handler to tell whether the call
is a system thread call or not. This change always sets this info to false
hence no functional change.

This change prepares management of system invocation proposed in a later
change.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Co-developed-by: Jens Wiklander <jens.wiklander@linaro.org>
Signed-off-by: Etienne Carriere <etienne.carriere@foss.st.com>
[jw: clarified that it's system thread calls]
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

firmware: arm_ffa: Split up ffa_ops into info, message and memory operations

In preparation to make memory operations accessible for a non
ffa_driver/device, it is better to split the ffa_ops into different
categories of operations: info, message and memory. The info and memory
are ffa_device independent and can be used without any associated
ffa_device from a non ffa_driver.

However, we don't export these info and memory APIs yet without the user.
The first users of these APIs can export them.

Link: https://lore.kernel.org/r/20220907145240.1683088-11-sudeep.holla@arm.com
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Signed-off-by: Sudeep Holla <sudeep.holla@arm.com>

firmware: arm_ffa: Rename ffa_dev_ops as ffa_ops

Except the message APIs, all other APIs are ffa_device independent and can
be used without any associated ffa_device from a non ffa_driver.

In order to reflect the same, just rename ffa_dev_ops as ffa_ops to
avoid any confusion or to keep it simple.

Link: https://lore.kernel.org/r/20220907145240.1683088-8-sudeep.holla@arm.com
Suggested-by: Sumit Garg <sumit.garg@linaro.org>
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Signed-off-by: Sudeep Holla <sudeep.holla@arm.com>

firmware: arm_ffa: Make memory apis ffa_device independent

There is a requirement to make memory APIs independent of the ffa_device.
One of the use-case is to have a common memory driver that manages the
memory for all the ffa_devices. That common memory driver won't be a
ffa_driver or won't have any ffa_device associated with it. So having
these memory APIs accessible without a ffa_device is needed and should
be possible as most of these are handled by the partition manager(SPM
or hypervisor).

Drop the ffa_device argument to the memory APIs and make them ffa_device
independent.

Link: https://lore.kernel.org/r/20220907145240.1683088-7-sudeep.holla@arm.com
Acked-by: Jens Wiklander <jens.wiklander@linaro.org>
Signed-off-by: Sudeep Holla <sudeep.holla@arm.com>

tee: optee: Drop ffa_ops in optee_ffa structure using ffa_dev->ops directly

Now that the ffa_device structure holds the pointer to ffa_dev_ops,
there is no need to obtain the same through ffa_dev_ops_get().

Just use the ffa_dev->ops directly. Since the ffa_device itself carries
ffa_dev_ops now, there is no need to keep a copy in optee_ffa structure.

Drop ffa_ops in the optee_ffa structure as it is not needed anymore.

Link: https://lore.kernel.org/r/20220907145240.1683088-3-sudeep.holla@arm.com
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Sudeep Holla <sudeep.holla@arm.com>

tee: optee: Use ffa_dev_get_drvdata to fetch driver_data

Due to lack of an helper like ffa_dev_get_drvdata, this driver was
fetching driver_data directly accessing the structure member. Now that
we have added an helper, just use the same instead.

Link: https://lore.kernel.org/r/20220429113946.2087145-4-sudeep.holla@arm.com
Signed-off-by: Sudeep Holla <sudeep.holla@arm.com>

optee: cache argument shared memory structs

Implements a cache to handle shared memory used to pass the argument
struct needed when doing a normal yielding call into secure world.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: add FF-A capability OPTEE_FFA_SEC_CAP_ARG_OFFSET

Adds the secure capability OPTEE_FFA_SEC_CAP_ARG_OFFSET to indicate that
OP-TEE with FF-A can support an argument struct at a non-zero offset into
a passed shared memory object.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: rename rpc_arg_count to rpc_param_count

Renames the field rpc_arg_count in struct optee to rpc_param_count.
Function parameter names and local variables are also renamed to match.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: optee: add missing mutext_destroy in optee_ffa_probe

The error handling code of optee_ffa_probe misses the mutex_destroy of
ffa.mutex when mutext_init succeeds.

Fix this by adding mutex_destory of ffa.mutex at the error handling part

Fixes: aceeafefff73 ("optee: use driver internal tee_context for some rpc")
Signed-off-by: Dongliang Mu <mudongliangabcd@gmail.com>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: add optee_pool_op_free_helper()

Adds a common helper function to free a tee_shm allocated using the
helper function optee_pool_op_alloc_helper().

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: replace tee_shm_alloc()

tee_shm_alloc() is replaced by three new functions,

tee_shm_alloc_user_buf() - for user mode allocations, replacing passing
the flags TEE_SHM_MAPPED | TEE_SHM_DMA_BUF

tee_shm_alloc_kernel_buf() - for kernel mode allocations, slightly
optimized compared to using the flags TEE_SHM_MAPPED | TEE_SHM_DMA_BUF.

tee_shm_alloc_priv_buf() - primarily for TEE driver internal use.

This also makes the interface easier to use as we can get rid of the
somewhat hard to use flags parameter.

The TEE subsystem and the TEE drivers are updated to use the new
functions instead.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: simplify shm pool handling

Replaces the shared memory pool based on two pools with a single pool.
The alloc() function pointer in struct tee_shm_pool_ops gets another
parameter, align. This makes it possible to make less than page aligned
allocations from the optional reserved shared memory pool while still
making user space allocations page aligned. With in practice unchanged
behaviour using only a single pool for bookkeeping.

The allocation algorithm in the static OP-TEE shared memory pool is
changed from best-fit to first-fit since only the latter supports an
alignment parameter. The best-fit algorithm was previously the default
choice and not a conscious one.

The optee and amdtee drivers are updated as needed to work with this
changed pool handling.

This also removes OPTEE_SHM_NUM_PRIV_PAGES which becomes obsolete with
this change as the private pages can be mixed with the payload pages.

The OP-TEE driver changes minimum alignment for argument struct from 8
bytes to 512 bytes. A typical OP-TEE private shm allocation is 224 bytes
(argument struct with 6 parameters, needed for open session). So with an
alignment of 512 well waste a bit more than 50%. Before this we had a
single page reserved for this so worst case usage compared to that would
be 3 pages instead of 1 page. However, this worst case only occurs if
there is a high pressure from multiple threads on secure world. All in
all this should scale up and down better than fixed boundaries.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: optee: fix error return code in probe function

If teedev_open() fails, probe function need return
error code.

Fixes: aceeafefff73 ("optee: use driver internal tee_context for some rpc")
Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Yang Yingliang <yangyingliang@huawei.com>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: use driver internal tee_context for some rpc

Adds a driver private tee_context by moving the tee_context in struct
optee_notif to struct optee. This tee_context was previously used when
doing internal calls to secure world to deliver notification.

The new driver internal tee_context is now also when allocating driver
private shared memory. This decouples the shared memory object from its
original tee_context. This is needed when the life time of such a memory
allocation outlives the client tee_context.

This patch fixes the problem described below:

The addition of a shutdown hook by commit f25889f93184 ("optee: fix tee out
of memory failure seen during kexec reboot") introduced a kernel shutdown
regression that can be triggered after running the OP-TEE xtest suites.

Once the shutdown hook is called it is not possible to communicate any more
with the supplicant process because the system is not scheduling task any
longer. Thus if the optee driver shutdown path receives a supplicant RPC
request from the OP-TEE we will deadlock the kernel's shutdown.

Fixes: f25889f93184 ("optee: fix tee out of memory failure seen during kexec reboot")
Fixes: 217e0250cccb ("tee: use reference counting for tee_context")
Reported-by: Lars Persson <larper@axis.com>
Cc: stable@vger.kernel.org
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: add error checks in optee_ffa_do_call_with_arg()

Adds error checking in optee_ffa_do_call_with_arg() for correctness.

Fixes: 4615e5a34b95 ("optee: add FF-A support")
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: separate notification functions

Renames struct optee_wait_queue to struct optee_notif and all related
functions to optee_notif_*().

The implementation is changed to allow sending a notification from an
atomic state, that is from the top half of an interrupt handler.

Waiting for keys is currently only used when secure world is waiting for
a mutex or condition variable. The old implementation could handle any
32-bit key while this new implementation is restricted to only 8 bits or
the maximum value 255. A upper value is needed since a bitmap is
allocated to allow an interrupt handler to only set a bit in case the
waiter hasn't had the time yet to allocate and register a completion.

The keys are currently only representing secure world threads which
number usually are never even close to 255 so it should be safe for now.
In future ABI updates the maximum value of the key will be communicated
while the driver is initializing.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: fix kfree NULL pointer

This patch fixes the following Coccinelle error:
drivers/tee/optee/ffa_abi.c: 877: ERROR optee is NULL but dereferenced.

If memory allocation fails, optee is null pointer. the code will goto err
and release optee.

Fixes: 4615e5a34b95 ("optee: add FF-A support")
Reported-by: Zeal Robot <zealci@zte.com.cn>
Signed-off-by: Lv Ruyi <lv.ruyi@zte.com.cn>
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
[jw: removed the redundant braces]
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: Fix spelling mistake "reclain" -> "reclaim"

There are spelling mistakes in pr_err error messages. Fix them.

Fixes: 4615e5a34b95 ("optee: add FF-A support")
Signed-off-by: Colin Ian King <colin.i.king@gmail.com>
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
[jw: added a fixes]
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: add FF-A support

Adds support for using FF-A [1] as transport to the OP-TEE driver.

Introduces struct optee_msg_param_fmem which carries all information
needed when OP-TEE is calling FFA_MEM_RETRIEVE_REQ to get the shared
memory reference mapped by the hypervisor in S-EL2. Register usage is
also updated to include the information needed.

The FF-A part of this driver is enabled if CONFIG_ARM_FFA_TRANSPORT is
enabled.

[1] https://developer.arm.com/documentation/den0077/latest
Acked-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>