optee: add missing description of RPC argument reference

Adds missing inline description comment for RPC optional arguments
reference.

Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/lkml/202310192021.fvb6JDOY-lkp@intel.com/
Signed-off-by: Etienne Carriere <etienne.carriere@foss.st.com>
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: add page list to kernel private shared memory

Until now has kernel private shared memory allocated as dynamic shared
memory (not from the static shared memory pool) been returned without a
list of physical pages on allocations via RPC. To support allocations
larger than one page add a list of physical pages.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: provide optee_do_bottom_half() as a common function

Provides optee_do_bottom_half() and optee_stop_async_notif() as common
functions callable from the FF-A ABI part of the driver too.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>

tee: optee: support tracking system threads

Adds support in the OP-TEE driver to keep track of reserved system
threads. The logic allows one OP-TEE thread to be reserved to TEE system
sessions.

The optee_cq_*() functions are updated to handle this if enabled,
that is when TEE describes how many thread context it supports
and when at least 1 session has registered as a system session
(using tee_client_system_session()).

For sake of simplicity, initialization of call queue management
is factorized into new helper function optee_cq_init().

The SMC ABI part of the driver enables this tracking, but the
FF-A ABI part does not.

Co-developed-by: Jens Wiklander <jens.wiklander@linaro.org>
Co-developed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Etienne Carriere <etienne.carriere@foss.st.com>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: optee: system thread call property

Adds an argument to do_call_with_arg() handler to tell whether the call
is a system thread call or not. This change always sets this info to false
hence no functional change.

This change prepares management of system invocation proposed in a later
change.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Co-developed-by: Jens Wiklander <jens.wiklander@linaro.org>
Signed-off-by: Etienne Carriere <etienne.carriere@foss.st.com>
[jw: clarified that it's system thread calls]
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: optee: Use kmemdup() to replace kmalloc + memcpy

Use kmemdup rather than duplicating its implementation.

./drivers/tee/optee/smc_abi.c:1542:12-19: WARNING opportunity for kmemdup.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Closes: https://bugzilla.openanolis.cn/show_bug.cgi?id=5480
Signed-off-by: Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: fix uninited async notif value

Fixes an uninitialized variable in irq_handler() that could lead to
unpredictable behavior in case OP-TEE fails to handle SMC function ID
OPTEE_SMC_GET_ASYNC_NOTIF_VALUE. This change ensures that in that case
get_async_notif_value() properly reports there are no notification
event.

Reported-by: kernel test robot <lkp@intel.com>
Link: https://lore.kernel.org/r/202304200755.OoiuclDZ-lkp@intel.com/
Reported-by: Dan Carpenter <error27@gmail.com>
Link: https://lore.kernel.org/all/d9b7f69b-c737-4cb3-8e74-79fe00c934f9@kili.mountain/
Fixes: 6749e69c4dad ("optee: add asynchronous notifications")
Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: optee: Add SMC for loading OP-TEE image

Adds an SMC call that will pass an OP-TEE binary image to EL3 and
instruct it to load it as the BL32 payload. This works in conjunction
with a feature added to Trusted Firmware for ARMv8 and above
architectures that supports this.

The main purpose of this change is to facilitate updating the OP-TEE
component on devices via a rootfs change rather than having to do a
firmware update. Further details are linked to in the Kconfig file.

Signed-off-by: Jeffrey Kardatzke <jkardatzke@chromium.org>
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jeffrey Kardatzke <jkardatzke@google.com>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: add per cpu asynchronous notification

Implements use of per-cpu irq for optee asynchronous notification.

Existing optee async notif implementation allows OP-TEE world to
raise an interrupt on which Linux optee driver will query some pending
events. This change allows the signaling interrupt to be a per-cpu
interrupt as with Arm GIC PPIs. Using a PPI instead of an SPI is useful
when no GIC lines are provisioned in the chip design and there are spare
PPI lines.

Cc: Jens Wiklander <jens.wiklander@linaro.org>
Cc: Sumit Garg <sumit.garg@linaro.org>
Cc: Marc Zyngier <maz@kernel.org>

Co-developed-by: Alexandre Torgue <alexandre.torgue@foss.st.com>
Signed-off-by: Alexandre Torgue <alexandre.torgue@foss.st.com>
Signed-off-by: Etienne Carriere <etienne.carriere@linaro.org>
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
[JW: fixing a spell error in @notif_pcpu_wq description]
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: smc_abi.c: fix wrong pointer passed to IS_ERR/PTR_ERR()

In optee_smc_do_call_with_arg() there is a code path when the argument
struct for RPC is passed appended to the primary argument struct. When
the address of the RPC struct is retrieved there's an invalid check for
success. It should be 'rpc_arg' pass to IS_ERR/PTR_ERR().

Fixes: ed8faf6c8f8c ("optee: add OPTEE_SMC_CALL_WITH_RPC_ARG and OPTEE_SMC_CALL_WITH_REGD_ARG")
Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Yang Yingliang <yangyingliang@huawei.com>
[jw: added background to the problem]
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: cache argument shared memory structs

Implements a cache to handle shared memory used to pass the argument
struct needed when doing a normal yielding call into secure world.

Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: add OPTEE_SMC_CALL_WITH_RPC_ARG and OPTEE_SMC_CALL_WITH_REGD_ARG

Adds OPTEE_SMC_CALL_WITH_RPC_ARG and OPTEE_SMC_CALL_WITH_REGD_ARG where
the struct optee_msg_arg to be used for RPC is appended in the memory
following the normal argument struct optee_msg_arg. This is an
optimization to avoid caching the RPC argument struct while still
maintaining similar performance as if it was cached.

OPTEE_SMC_CALL_WITH_REGD_ARG optimized one step further by using a
registered shared memory object instead. It's in other aspects identical
to OPTEE_SMC_CALL_WITH_RPC_ARG.

The presence of OPTEE_SMC_CALL_WITH_RPC_ARG and
OPTEE_SMC_CALL_WITH_REGD_ARG is indicated by the new
OPTEE_SMC_SEC_CAP_RPC_ARG bit returned by
OPTEE_SMC_EXCHANGE_CAPABILITIES. OPTEE_SMC_EXCHANGE_CAPABILITIES also
reports the number of arguments that the RPC argument struct must have
room for.

OPTEE_SMC_CALL_WITH_RPC_ARG and OPTEE_SMC_CALL_WITH_ARG can be used
interleaved with difference that when OPTEE_SMC_CALL_WITH_RPC_ARG is
used the RPC argument struct to be used is the one appended to the
normal argument struct. The same is true for
OPTEE_SMC_CALL_WITH_REGD_ARG.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: refactor TEE_SHM_* flags

Removes the redundant TEE_SHM_DMA_BUF, TEE_SHM_EXT_DMA_BUF,
TEE_SHM_MAPPED and TEE_SHM_KERNEL_MAPPED flags.

TEE_SHM_REGISTER is renamed to TEE_SHM_DYNAMIC in order to better
match its usage.

Assigns new values to the remaining flags to void gaps.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: add optee_pool_op_free_helper()

Adds a common helper function to free a tee_shm allocated using the
helper function optee_pool_op_alloc_helper().

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: replace tee_shm_alloc()

tee_shm_alloc() is replaced by three new functions,

tee_shm_alloc_user_buf() - for user mode allocations, replacing passing
the flags TEE_SHM_MAPPED | TEE_SHM_DMA_BUF

tee_shm_alloc_kernel_buf() - for kernel mode allocations, slightly
optimized compared to using the flags TEE_SHM_MAPPED | TEE_SHM_DMA_BUF.

tee_shm_alloc_priv_buf() - primarily for TEE driver internal use.

This also makes the interface easier to use as we can get rid of the
somewhat hard to use flags parameter.

The TEE subsystem and the TEE drivers are updated to use the new
functions instead.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: simplify shm pool handling

Replaces the shared memory pool based on two pools with a single pool.
The alloc() function pointer in struct tee_shm_pool_ops gets another
parameter, align. This makes it possible to make less than page aligned
allocations from the optional reserved shared memory pool while still
making user space allocations page aligned. With in practice unchanged
behaviour using only a single pool for bookkeeping.

The allocation algorithm in the static OP-TEE shared memory pool is
changed from best-fit to first-fit since only the latter supports an
alignment parameter. The best-fit algorithm was previously the default
choice and not a conscious one.

The optee and amdtee drivers are updated as needed to work with this
changed pool handling.

This also removes OPTEE_SHM_NUM_PRIV_PAGES which becomes obsolete with
this change as the private pages can be mixed with the payload pages.

The OP-TEE driver changes minimum alignment for argument struct from 8
bytes to 512 bytes. A typical OP-TEE private shm allocation is 224 bytes
(argument struct with 6 parameters, needed for open session). So with an
alignment of 512 well waste a bit more than 50%. Before this we had a
single page reserved for this so worst case usage compared to that would
be 3 pages instead of 1 page. However, this worst case only occurs if
there is a high pressure from multiple threads on secure world. All in
all this should scale up and down better than fixed boundaries.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: optee: fix error return code in probe function

If teedev_open() fails, probe function need return
error code.

Fixes: aceeafefff73 ("optee: use driver internal tee_context for some rpc")
Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: Yang Yingliang <yangyingliang@huawei.com>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: use driver internal tee_context for some rpc

Adds a driver private tee_context by moving the tee_context in struct
optee_notif to struct optee. This tee_context was previously used when
doing internal calls to secure world to deliver notification.

The new driver internal tee_context is now also when allocating driver
private shared memory. This decouples the shared memory object from its
original tee_context. This is needed when the life time of such a memory
allocation outlives the client tee_context.

This patch fixes the problem described below:

The addition of a shutdown hook by commit f25889f93184 ("optee: fix tee out
of memory failure seen during kexec reboot") introduced a kernel shutdown
regression that can be triggered after running the OP-TEE xtest suites.

Once the shutdown hook is called it is not possible to communicate any more
with the supplicant process because the system is not scheduling task any
longer. Thus if the optee driver shutdown path receives a supplicant RPC
request from the OP-TEE we will deadlock the kernel's shutdown.

Fixes: f25889f93184 ("optee: fix tee out of memory failure seen during kexec reboot")
Fixes: 217e0250cccb ("tee: use reference counting for tee_context")
Reported-by: Lars Persson <larper@axis.com>
Cc: stable@vger.kernel.org
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

tee: optee: do not check memref size on return from Secure World

Commit c650b8dc7a79 ("tee: optee: do not check memref size on return
from Secure World") was mistakenly lost in commit 4602c5842f64 ("optee:
refactor driver with internal callbacks"). Remove the unwanted code
again.

Fixes: 4602c5842f64 ("optee: refactor driver with internal callbacks")
Signed-off-by: Jerome Forissier <jerome@forissier.org>
Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: Fix NULL but dereferenced coccicheck error

Eliminate the following coccicheck warning:
./drivers/tee/optee/smc_abi.c:1508:12-15: ERROR: optee is NULL but
dereferenced.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Fixes: 6749e69c4dad ("optee: add asynchronous notifications")
Signed-off-by: Yang Li <yang.lee@linux.alibaba.com>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: add asynchronous notifications

Adds support for asynchronous notifications from secure world to normal
world. This allows a design with a top half and bottom half type of
driver where the top half runs in secure interrupt context and a
notifications tells normal world to schedule a yielding call to do the
bottom half processing.

The protocol is defined in optee_msg.h optee_rpc_cmd.h and optee_smc.h.

A notification consists of a 32-bit value which normal world can
retrieve using a fastcall into secure world. The value
OPTEE_SMC_ASYNC_NOTIF_VALUE_DO_BOTTOM_HALF (0) has a special meaning.
When this value is sent it means that normal world is supposed to make a
yielding call OPTEE_MSG_CMD_DO_BOTTOM_HALF.

Notification capability is negotiated while the driver is initialized.
If both sides supports these notifications then they are enabled.

An interrupt is used to notify the driver that there are asynchronous
notifications pending. The maximum needed notification value is
communicated at this stage. This allows scaling up when needed.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: separate notification functions

Renames struct optee_wait_queue to struct optee_notif and all related
functions to optee_notif_*().

The implementation is changed to allow sending a notification from an
atomic state, that is from the top half of an interrupt handler.

Waiting for keys is currently only used when secure world is waiting for
a mutex or condition variable. The old implementation could handle any
32-bit key while this new implementation is restricted to only 8 bits or
the maximum value 255. A upper value is needed since a bitmap is
allocated to allow an interrupt handler to only set a bit in case the
waiter hasn't had the time yet to allocate and register a completion.

The keys are currently only representing secure world threads which
number usually are never even close to 255 so it should be safe for now.
In future ABI updates the maximum value of the key will be communicated
while the driver is initializing.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: Suppress false positive kmemleak report in optee_handle_rpc()

We observed the following kmemleak report:
unreferenced object 0xffff000007904500 (size 128):
comm "swapper/0", pid 1, jiffies 4294892671 (age 44.036s)
hex dump (first 32 bytes):
00 47 90 07 00 00 ff ff 60 00 c0 ff 00 00 00 00 .G......`.......
60 00 80 13 00 80 ff ff a0 00 00 00 00 00 00 00 `...............
backtrace:
[<000000004c12b1c7>] kmem_cache_alloc+0x1ac/0x2f4
[<000000005d23eb4f>] tee_shm_alloc+0x78/0x230
[<00000000794dd22c>] optee_handle_rpc+0x60/0x6f0
[<00000000d9f7c52d>] optee_do_call_with_arg+0x17c/0x1dc
[<00000000c35884da>] optee_open_session+0x128/0x1ec
[<000000001748f2ff>] tee_client_open_session+0x28/0x40
[<00000000aecb5389>] optee_enumerate_devices+0x84/0x2a0
[<000000003df18bf1>] optee_probe+0x674/0x6cc
[<000000003a4a534a>] platform_drv_probe+0x54/0xb0
[<000000000c51ce7d>] really_probe+0xe4/0x4d0
[<000000002f04c865>] driver_probe_device+0x58/0xc0
[<00000000b485397d>] device_driver_attach+0xc0/0xd0
[<00000000c835f0df>] __driver_attach+0x84/0x124
[<000000008e5a429c>] bus_for_each_dev+0x70/0xc0
[<000000001735e8a8>] driver_attach+0x24/0x30
[<000000006d94b04f>] bus_add_driver+0x104/0x1ec

This is not a memory leak because we pass the share memory pointer
to secure world and would get it from secure world before releasing it.

Signed-off-by: Xiaolei Wang <xiaolei.wang@windriver.com>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

optee: smc_abi.c: add missing #include <linux/mm.h>

Adds missing #include <linux/mm.h> drivers/tee/optee/smc_abi.c to fix
compile errors like:
drivers/tee/optee/smc_abi.c:405:15: error: implicit
declaration of function 'page_to_section'
[-Werror,-Wimplicit-function-declaration]
optee_page = page_to_phys(*pages) +
^
arch/arm/include/asm/memory.h:148:43: note: expanded from
macro 'page_to_phys'
^
include/asm-generic/memory_model.h:52:21: note: expanded
from macro 'page_to_pfn'
^
include/asm-generic/memory_model.h:35:14: note: expanded
from macro '__page_to_pfn'
int __sec = page_to_section(__pg); \
^
drivers/tee/optee/smc_abi.c:405:15: note: did you mean
'__nr_to_section'?
arch/arm/include/asm/memory.h:148:43: note: expanded from
macro 'page_to_phys'
^
include/asm-generic/memory_model.h:52:21: note: expanded
from macro 'page_to_pfn'
^
include/asm-generic/memory_model.h:35:14: note: expanded
from macro '__page_to_pfn'
int __sec = page_to_section(__pg); \
^
include/linux/mmzone.h:1365:35: note: '__nr_to_section'
declared here
static inline struct mem_section *__nr_to_section(unsigned long nr)

Fixes: c51a564a5b48 ("optee: isolate smc abi")
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Link: https://lore.kernel.org/r/20211021125539.3858495-1-jens.wiklander@linaro.org'
Signed-off-by: Arnd Bergmann <arnd@arndb.de>

optee: isolate smc abi

Isolate the ABI based on raw SMCs. Code specific to the raw SMC ABI is
moved into smc_abi.c. This makes room for other ABIs with a clear
separation.

The driver changes to use module_init()/module_exit() instead of
module_platform_driver(). The platform_driver_register() and
platform_driver_unregister() functions called directly to keep the same
behavior. This is needed because module_platform_driver() is based on
module_driver() which can only be used once in a module.

A function optee_rpc_cmd() is factored out from the function
handle_rpc_func_cmd() to handle the ABI independent part of RPC
processing.

This patch is not supposed to change the driver behavior, it's only a
matter of reorganizing the code.

Reviewed-by: Sumit Garg <sumit.garg@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>