History log of /linux-master/drivers/media/platform/verisilicon/hantro_v4l2.c
Revision Date Author Comments
# 6c0d9e12 09-Nov-2023 Paul Kocialkowski <paul.kocialkowski@bootlin.com>

media: verisilicon: Hook the (TRY_)DECODER_CMD stateless ioctls

The (TRY_)DECODER_CMD ioctls are used to support flushing when holding
capture buffers is supported. This is the case of this driver but the
ioctls were never hooked to the ioctl ops.

Add them to correctly support flushing.

Fixes: 340ce50f75a6 ("media: hantro: Enable HOLD_CAPTURE_BUF for H.264")
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# bbd267da 09-Nov-2023 Benjamin Gaignard <benjamin.gaignard@collabora.com>

media: verisilicon: vp9: Allow to change resolution while streaming

Remove all checks that prohibit to set a new format while streaming.
This allow to change dynamically the resolution if the pixel format
remains the same.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Andrzej Pietrasiewicz <andrzej.p@collabora.com>
CC: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
CC: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# 26711491 09-Nov-2023 Benjamin Gaignard <benjamin.gaignard@collabora.com>

media: verisilicon: Refactor postprocessor to store more buffers

Since vb2 queue can store more than VB2_MAX_FRAME buffers, the
postprocessor buffer storage must be capable to store more buffers too.
Change static dec_q array to allocated array to be capable to store
up to queue 'max_num_buffers'.
Keep allocating queue 'num_buffers' at queue setup time but also allows
to allocate postprocessors buffers on the fly.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Andrzej Pietrasiewicz <andrzej.p@collabora.com>
CC: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
CC: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# b3b4c9d3 06-Jul-2023 Michael Tretter <m.tretter@pengutronix.de>

media: verisilicon: Fix TRY_FMT on encoder OUTPUT

Commit f100ce3bbd6a ("media: verisilicon: Fix crash when probing
encoder") removed vpu_fmt from hantro_try_fmt(), since it was
initialized from vpu_dst_fmt, which may not be initialized, when TRY_FMT
is called. It was replaced by fmt, which is found using the pixelformat.

For the encoder, this changed the fmt to contain the raw format instead
of the coded format. The format constraints as of fmt->frmsize are only
valid for the coded format and are 0 for the raw formats. Therefore, the
size of a encoder OUTPUT device is constrained to 0 and the
v4l2-compliance tests for G_FMT, TRY_FMT, and SET_FMT fail.

Bring back vpu_fmt to use the coded format on an encoder OUTPUT device,
but initialize it using the currently set pixelformat on dst_fmt, which
is the coded format on an encoder.

Fixes: f100ce3bbd6a ("media: verisilicon: Fix crash when probing encoder")
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>


# 80c7373a 03-May-2023 Benjamin Gaignard <benjamin.gaignard@collabora.com>

media: verisilicon: Conditionally ignore native formats

AV1 film grain feature requires to use the postprocessor to produce
valid frames. In such case the driver shouldn't propose native pixels
format but only post-processed pixels format.
Additionally if when setting a control a value could change capture
queue pixels formats it is needed to call hantro_reset_raw_fmt().

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# 1b9ef274 03-May-2023 Benjamin Gaignard <benjamin.gaignard@collabora.com>

media: verisilicon: Compute motion vectors size for AV1 frames

Compute the additional space required to store motion vectors at
the end of the frames buffers.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# 7040ed4e 03-May-2023 Benjamin Gaignard <benjamin.gaignard@collabora.com>

media: verisilicon: Check AV1 bitstreams bit depth

The driver supports 8 and 10 bits bitstreams, make sure to discard
other cases.
It could happens that userland test if V4L2_CID_STATELESS_AV1_SEQUENCE
exists without setting bit_depth field in this case use
HANTRO_DEFAULT_BIT_DEPTH value.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# aa1e34c8 03-May-2023 Benjamin Gaignard <benjamin.gaignard@collabora.com>

media: verisilicon: Get bit depth for V4L2_PIX_FMT_NV15_4L4

Let's the driver knows that V4L2_PIX_FMT_NV15_4L4 is a 10bits
pixel format.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# 14b4bd01 23-May-2023 Benjamin Gaignard <benjamin.gaignard@collabora.com>

media: verisilicon: Additional fix for the crash when opening the driver

This fixes the following issue observed on Odroid-M1 board:

Unable to handle kernel NULL pointer dereference at virtual address 0000000000000008
Mem abort info:
...
Modules linked in: crct10dif_ce hantro_vpu snd_soc_simple_card snd_soc_simple_card_utils v4l2_vp9 v4l2_h264 rockchip_saradc v4l2_mem2mem videobuf2_dma_contig videobuf2_memops rtc_rk808 videobuf2_v4l2 industrialio_triggered_buffer rockchip_thermal dwmac_rk stmmac_platform stmmac videodev kfifo_buf display_connector videobuf2_common pcs_xpcs mc rockchipdrm analogix_dp dw_mipi_dsi dw_hdmi drm_display_helper panfrost drm_shmem_helper gpu_sched ip_tables x_tables ipv6
CPU: 3 PID: 176 Comm: v4l_id Not tainted 6.3.0-rc7-next-20230420 #13481
Hardware name: Hardkernel ODROID-M1 (DT)
pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : hantro_try_fmt+0xa0/0x278 [hantro_vpu]
lr : hantro_try_fmt+0x94/0x278 [hantro_vpu]
...
Call trace:
hantro_try_fmt+0xa0/0x278 [hantro_vpu]
hantro_set_fmt_out+0x3c/0x298 [hantro_vpu]
hantro_reset_raw_fmt+0x98/0x128 [hantro_vpu]
hantro_set_fmt_cap+0x240/0x254 [hantro_vpu]
hantro_reset_encoded_fmt+0x94/0xcc [hantro_vpu]
hantro_reset_fmts+0x18/0x38 [hantro_vpu]
hantro_open+0xd4/0x20c [hantro_vpu]
v4l2_open+0x80/0x120 [videodev]
chrdev_open+0xc0/0x22c
do_dentry_open+0x13c/0x48c
vfs_open+0x2c/0x38
path_openat+0x550/0x934
do_filp_open+0x80/0x12c
do_sys_openat2+0xb4/0x168
__arm64_sys_openat+0x64/0xac
invoke_syscall+0x48/0x114
el0_svc_common+0x100/0x120
do_el0_svc+0x3c/0xa8
el0_svc+0x40/0xa8
el0t_64_sync_handler+0xb8/0xbc
el0t_64_sync+0x190/0x194
Code: 97fc8a7f f940aa80 52864a61 72a686c1 (b9400800)
---[ end trace 0000000000000000 ]---

Fixes: db6f68b51e5c ("media: verisilicon: Do not set context src/dst formats in reset functions")
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Tested-by: Michael Tretter <m.tretter@pengutronix.de>
Tested-by: Diederik de Haas <didi.debian@cknow.org>
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# f100ce3b 12-Apr-2023 Benjamin Gaignard <benjamin.gaignard@collabora.com>

media: verisilicon: Fix crash when probing encoder

ctx->vpu_dst_fmt is no more initialized before calling hantro_try_fmt()
so assigne it to vpu_fmt led to crash the kernel.
Like for decoder case use 'fmt' as format for encoder and clean up
the code.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
Fixes: db6f68b51e5c ("media: verisilicon: Do not set context src/dst formats in reset functions")
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>


# 3c32d94c 20-Feb-2023 Benjamin Gaignard <benjamin.gaignard@collabora.com>

media: verisilicon: Do not change context bit depth before validating the format

It is needed to check if the proposed pixels format is valid before
updating context bit depth and other internal states.
Stop using ctx->bit_depth to check format depth match and return
result to the caller.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# 3b93a6f0 20-Feb-2023 Benjamin Gaignard <benjamin.gaignard@collabora.com>

media: verisilicon: Do not use ctx fields as format storage when resetting

Source and destination pixel formats fields of context structure should
not be used as storage when resetting the format.
Use local variables instead and let hantro_set_fmt_out() and
hantro_set_fmt_cap() set them correctly later.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# db6f68b5 20-Feb-2023 Benjamin Gaignard <benjamin.gaignard@collabora.com>

media: verisilicon: Do not set context src/dst formats in reset functions

Setting context source and destination formats should only be done
in hantro_set_fmt_out() and hantro_set_fmt_cap() after check that
the targeted queue is not busy.
Remove these calls from hantro_reset_encoded_fmt() and
hantro_reset_raw_fmt() to clean the driver.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# c9ca3b53 25-Dec-2022 Chen-Yu Tsai <wenst@chromium.org>

media: hantro: Use core-generated bus_info value

The Hantro driver uses a hardcoded value for the bus_info field in the
media device and |struct v4l2_capability|. This worked well when there
was just one device. However with the iMX.8 series we are now seeing
two Hantro blocks on the same chip. The static bus_info is no longer
sufficient for differentiating devices.

Since commit f2d8b6917f3b ("media: v4l: ioctl: Set bus_info in
v4l_querycap()"), the V4L2 core provides a default value for the
bus_info field for platform and PCI devices. This value will match
the default value for media devices added by commit cef699749f37
("media: mc: Set bus_info in media_device_init()"). These defaults
are stable and device-specific.

Drop the static bus_info values from the hantro driver and use the
defaults.

Signed-off-by: Chen-Yu Tsai <wenst@chromium.org>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# 29bd4267 23-Dec-2022 Nicolas Dufresne <nicolas.dufresne@collabora.com>

media: hantro: Fix JPEG encoder ENUM_FRMSIZE on RK3399

Since 79c987de8b354, enumerating framesize on format set with "MODE_NONE"
(any raw formats) is reporting an invalid frmsize.

Size: Stepwise 0x0 - 0x0 with step 0/0

Before this change, the driver would return EINVAL, which is also invalid
but worked in GStreamer. The original intent was not to implement it, hence
the -ENOTTY return in this change. While drivers should implement
ENUM_FRMSIZE for all formats and queues, this change is limited in scope to
fix the regression.

This fixes taking picture in Gnome Cheese software, or any software using
GSteamer to encode JPEG with hardware acceleration.

Fixes: 79c987de8b35 ("media: hantro: Use post processor scaling capacities")
Reported-by: Robert Mader <robert.mader@collabora.com>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Tested-by: Robert Mader <robert.mader@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>


# fbb6c848 18-Jul-2022 Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>

media: destage Hantro VPU driver

The Hantro mainline driver has been used in production
since several years and was only kept as a staging driver
due the stateless CODEC controls.

Now that all the stateless CODEC controls have been moved
out of staging, graduate the driver as well.

Signed-off-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>