media: staging: ipu3-imgu: Set fields before media_entity_pads_init()

The imgu driver fails to probe with the following message because it
does not set the pad's flags before calling media_entity_pads_init().

[ 14.596315] ipu3-imgu 0000:00:05.0: failed initialize subdev media entity (-22)
[ 14.596322] ipu3-imgu 0000:00:05.0: failed to register subdev0 ret (-22)
[ 14.596327] ipu3-imgu 0000:00:05.0: failed to register pipes (-22)
[ 14.596331] ipu3-imgu 0000:00:05.0: failed to create V4L2 devices (-22)

Fix the initialization order so that the driver probe succeeds. The ops
initialization is also moved together for readability.

Fixes: a0ca1627b450 ("media: staging/intel-ipu3: Add v4l2 driver based on media framework")
Cc: <stable@vger.kernel.org> # 6.7
Cc: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: Hidenori Kobayashi <hidenorik@chromium.org>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

media: videobuf2: core: Rename min_buffers_needed field in vb2_queue

Rename min_buffers_needed into min_queued_buffers and update
the documentation about it.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
[hverkuil: Drop the change where min_queued_buffers + 1 buffers would be]
[hverkuil: allocated. Now this patch only renames this field instead of making]
[hverkuil: a functional change as well.]
[hverkuil: Renamed 3 remaining min_buffers_needed occurrences.]

media: v4l: subdev: Switch to stream-aware state functions

Switch all drivers accessing sub-device state to use the stream-aware
functions. We will soon remove the old ones.

This patch has been generated using the following Coccinelle script:

---------8<------------
@@
expression E1, E2, E3;

@@

- v4l2_subdev_get_pad_format(E1, E2, E3)
+ v4l2_subdev_state_get_format(E2, E3)

@@
expression E1, E2, E3;

@@

- v4l2_subdev_get_pad_crop(E1, E2, E3)
+ v4l2_subdev_state_get_crop(E2, E3)

@@
expression E1, E2, E3;

@@

- v4l2_subdev_get_pad_compose(E1, E2, E3)
+ v4l2_subdev_state_get_compose(E2, E3)

@@
expression E1, E2, E3;

@@

- v4l2_subdev_get_try_format(E1, E2, E3)
+ v4l2_subdev_state_get_format(E2, E3)

@@
expression E1, E2, E3;

@@

- v4l2_subdev_get_try_crop(E1, E2, E3)
+ v4l2_subdev_state_get_crop(E2, E3)

@@
expression E1, E2, E3;

@@

- v4l2_subdev_get_try_compose(E1, E2, E3)
+ v4l2_subdev_state_get_compose(E2, E3)
---------8<------------

Additionally drivers/media/i2c/s5k5baf.c and
drivers/media/platform/samsung/s3c-camif/camif-capture.c have been
manually changed as Coccinelle didn't. Further local variables have been
removed as they became unused as a result of the other changes.

Also Coccinelle introduced indentation by space in files
drivers/media/i2c/st-mipid02.c and
drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c. This has been also
corrected.

The diff from Coccinelle-generated changes are:

> diff --git b/drivers/media/i2c/imx319.c a/drivers/media/i2c/imx319.c
> index e549692ff478..420984382173 100644
> --- b/drivers/media/i2c/imx319.c
> +++ a/drivers/media/i2c/imx319.c
> @@ -2001,7 +2001,6 @@ static int imx319_do_get_pad_format(struct imx319 *imx319,
> struct v4l2_subdev_format *fmt)
> {
> struct v4l2_mbus_framefmt *framefmt;
> - struct v4l2_subdev *sd = &imx319->sd;
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> diff --git b/drivers/media/i2c/imx355.c a/drivers/media/i2c/imx355.c
> index 96bdde685d65..e1b1d2fc79dd 100644
> --- b/drivers/media/i2c/imx355.c
> +++ a/drivers/media/i2c/imx355.c
> @@ -1299,7 +1299,6 @@ static int imx355_do_get_pad_format(struct imx355 *imx355,
> struct v4l2_subdev_format *fmt)
> {
> struct v4l2_mbus_framefmt *framefmt;
> - struct v4l2_subdev *sd = &imx355->sd;
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> diff --git b/drivers/media/i2c/ov08x40.c a/drivers/media/i2c/ov08x40.c
> index ca799bbcfdb7..abbb0b774d43 100644
> --- b/drivers/media/i2c/ov08x40.c
> +++ a/drivers/media/i2c/ov08x40.c
> @@ -2774,7 +2774,6 @@ static int ov08x40_do_get_pad_format(struct ov08x40 *ov08x,
> struct v4l2_subdev_format *fmt)
> {
> struct v4l2_mbus_framefmt *framefmt;
> - struct v4l2_subdev *sd = &ov08x->sd;
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> diff --git b/drivers/media/i2c/ov13858.c a/drivers/media/i2c/ov13858.c
> index 7816d9787c61..09387e335d80 100644
> --- b/drivers/media/i2c/ov13858.c
> +++ a/drivers/media/i2c/ov13858.c
> @@ -1316,7 +1316,6 @@ static int ov13858_do_get_pad_format(struct ov13858 *ov13858,
> struct v4l2_subdev_format *fmt)
> {
> struct v4l2_mbus_framefmt *framefmt;
> - struct v4l2_subdev *sd = &ov13858->sd;
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> diff --git b/drivers/media/i2c/ov13b10.c a/drivers/media/i2c/ov13b10.c
> index 268cd4b03f9c..c06411d5ee2b 100644
> --- b/drivers/media/i2c/ov13b10.c
> +++ a/drivers/media/i2c/ov13b10.c
> @@ -1001,7 +1001,6 @@ static int ov13b10_do_get_pad_format(struct ov13b10 *ov13b,
> struct v4l2_subdev_format *fmt)
> {
> struct v4l2_mbus_framefmt *framefmt;
> - struct v4l2_subdev *sd = &ov13b->sd;
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> diff --git b/drivers/media/i2c/s5c73m3/s5c73m3-core.c a/drivers/media/i2c/s5c73m3/s5c73m3-core.c
> index 47605e36bc60..8f9b5713daf7 100644
> --- b/drivers/media/i2c/s5c73m3/s5c73m3-core.c
> +++ a/drivers/media/i2c/s5c73m3/s5c73m3-core.c
> @@ -819,7 +819,6 @@ static void s5c73m3_oif_try_format(struct s5c73m3 *state,
> struct v4l2_subdev_format *fmt,
> const struct s5c73m3_frame_size **fs)
> {
> - struct v4l2_subdev *sd = &state->sensor_sd;
> u32 code;
>
> switch (fmt->pad) {
> diff --git a/drivers/media/i2c/s5k5baf.c b/drivers/media/i2c/s5k5baf.c
> index 67da2045f543..03ccfb0e1e11 100644
> --- a/drivers/media/i2c/s5k5baf.c
> +++ b/drivers/media/i2c/s5k5baf.c
> @@ -1472,14 +1472,11 @@ static int s5k5baf_set_selection(struct v4l2_subdev *sd,
>
> if (sel->which == V4L2_SUBDEV_FORMAT_TRY) {
> rects = (struct v4l2_rect * []) {
> - &s5k5baf_cis_rect,
> - v4l2_subdev_get_try_crop(sd, sd_state,
> - PAD_CIS),
> - v4l2_subdev_get_try_compose(sd, sd_state,
> - PAD_CIS),
> - v4l2_subdev_get_try_crop(sd, sd_state,
> - PAD_OUT)
> - };
> + &s5k5baf_cis_rect,
> + v4l2_subdev_state_get_crop(sd_state, PAD_CIS),
> + v4l2_subdev_state_get_compose(sd_state, PAD_CIS),
> + v4l2_subdev_state_get_crop(sd_state, PAD_OUT)
> + };
> s5k5baf_set_rect_and_adjust(rects, rtype, &sel->r);
> return 0;
> }
> diff --git b/drivers/media/platform/samsung/s3c-camif/camif-capture.c a/drivers/media/platform/samsung/s3c-camif/camif-capture.c
> index 295e083f38e8..be58260ea67e 100644
> --- b/drivers/media/platform/samsung/s3c-camif/camif-capture.c
> +++ a/drivers/media/platform/samsung/s3c-camif/camif-capture.c
> @@ -1216,7 +1216,7 @@ static int s3c_camif_subdev_get_fmt(struct v4l2_subdev *sd,
> struct v4l2_mbus_framefmt *mf = &fmt->format;
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> - mf = v4l2_subdev_get_try_format(sd, sd_state, fmt->pad);
> + mf = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> fmt->format = *mf;
> return 0;
> }
> @@ -1305,7 +1305,7 @@ static int s3c_camif_subdev_set_fmt(struct v4l2_subdev *sd,
> __camif_subdev_try_format(camif, mf, fmt->pad);
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> - mf = v4l2_subdev_get_try_format(sd, sd_state, fmt->pad);
> + mf = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> *mf = fmt->format;
> mutex_unlock(&camif->lock);
> return 0;
> diff --git b/drivers/media/platform/ti/cal/cal-camerarx.c a/drivers/media/platform/ti/cal/cal-camerarx.c
> index cea454ed9c20..61433744c6c4 100644
> --- b/drivers/media/platform/ti/cal/cal-camerarx.c
> +++ a/drivers/media/platform/ti/cal/cal-camerarx.c
> @@ -621,8 +621,6 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
> struct v4l2_subdev_state *state,
> struct v4l2_subdev_mbus_code_enum *code)
> {
> - struct cal_camerarx *phy = to_cal_camerarx(sd);
> -
> /* No transcoding, source and sink codes must match. */
> if (cal_rx_pad_is_source(code->pad)) {
> struct v4l2_mbus_framefmt *fmt;
> diff --git b/drivers/staging/media/imx/imx-ic-prp.c a/drivers/staging/media/imx/imx-ic-prp.c
> index dd558fac6477..61d69f19657e 100644
> --- b/drivers/staging/media/imx/imx-ic-prp.c
> +++ a/drivers/staging/media/imx/imx-ic-prp.c
> @@ -82,8 +82,6 @@ static struct v4l2_mbus_framefmt *
> __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_state *sd_state,
> unsigned int pad, enum v4l2_subdev_format_whence which)
> {
> - struct imx_ic_priv *ic_priv = priv->ic_priv;
> -
> if (which == V4L2_SUBDEV_FORMAT_TRY)
> return v4l2_subdev_state_get_format(sd_state, pad);
> else
> diff --git b/drivers/staging/media/imx/imx-ic-prpencvf.c a/drivers/staging/media/imx/imx-ic-prpencvf.c
> index 02db7dbb884b..ec73c901079e 100644
> --- b/drivers/staging/media/imx/imx-ic-prpencvf.c
> +++ a/drivers/staging/media/imx/imx-ic-prpencvf.c
> @@ -790,8 +790,6 @@ static struct v4l2_mbus_framefmt *
> __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_state *sd_state,
> unsigned int pad, enum v4l2_subdev_format_whence which)
> {
> - struct imx_ic_priv *ic_priv = priv->ic_priv;
> -
> if (which == V4L2_SUBDEV_FORMAT_TRY)
> return v4l2_subdev_state_get_format(sd_state, pad);
> else
> diff --git a/drivers/media/i2c/st-mipid02.c b/drivers/media/i2c/st-mipid02.c
> index 9c9361354c00..b08a249b5fdd 100644
> --- a/drivers/media/i2c/st-mipid02.c
> +++ b/drivers/media/i2c/st-mipid02.c
> @@ -751,7 +751,7 @@ static void mipid02_set_fmt_source(struct v4l2_subdev *sd,
> format->format = bridge->fmt;
> else
> format->format = *v4l2_subdev_state_get_format(sd_state,
> - MIPID02_SINK_0);
> + MIPID02_SINK_0);
>
> /* but code may need to be converted */
> format->format.code = serial_to_parallel_code(format->format.code);
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
> index 117912d3bfbd..96353648c032 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
> @@ -319,7 +319,7 @@ static void rkisp1_isp_start(struct rkisp1_isp *isp,
> rkisp1_write(rkisp1, RKISP1_CIF_ISP_CTRL, val);
>
> src_fmt = v4l2_subdev_state_get_format(sd_state,
> - RKISP1_ISP_PAD_SOURCE_VIDEO);
> + RKISP1_ISP_PAD_SOURCE_VIDEO);
> src_info = rkisp1_mbus_info_get_by_code(src_fmt->code);
>
> if (src_info->pixel_enc != V4L2_PIXEL_ENC_BAYER)
> @@ -475,9 +475,9 @@ static void rkisp1_isp_set_src_fmt(struct rkisp1_isp *isp,
> sink_fmt = v4l2_subdev_state_get_format(sd_state,
> RKISP1_ISP_PAD_SINK_VIDEO);
> src_fmt = v4l2_subdev_state_get_format(sd_state,
> - RKISP1_ISP_PAD_SOURCE_VIDEO);
> + RKISP1_ISP_PAD_SOURCE_VIDEO);
> src_crop = v4l2_subdev_state_get_crop(sd_state,
> - RKISP1_ISP_PAD_SOURCE_VIDEO);
> + RKISP1_ISP_PAD_SOURCE_VIDEO);
>
> /*
> * Media bus code. The ISP can operate in pass-through mode (Bayer in,

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

ipu3-imgu: Fix NULL pointer dereference in imgu_subdev_set_selection()

Calling v4l2_subdev_get_try_crop() and v4l2_subdev_get_try_compose()
with a subdev state of NULL leads to a NULL pointer dereference. This
can currently happen in imgu_subdev_set_selection() when the state
passed in is NULL, as this method first gets pointers to both the "try"
and "active" states and only then decides which to use.

The same issue has been addressed for imgu_subdev_get_selection() with
commit 30d03a0de650 ("ipu3-imgu: Fix NULL pointer dereference in active
selection access"). However the issue still persists in
imgu_subdev_set_selection().

Therefore, apply a similar fix as done in the aforementioned commit to
imgu_subdev_set_selection(). To keep things a bit cleaner, introduce
helper functions for "crop" and "compose" access and use them in both
imgu_subdev_set_selection() and imgu_subdev_get_selection().

Fixes: 0d346d2a6f54 ("media: v4l2-subdev: add subdev-wide state struct")
Cc: stable@vger.kernel.org # for v5.14 and later
Signed-off-by: Maximilian Luz <luzmaximilian@gmail.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

media: drivers: use video device pipeline start/stop

Convert the media drivers to use video device based pipeline start/stop
where possible.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

media: ipu3-imgu: Fix NULL pointer dereference in active selection access

What the IMGU driver did was that it first acquired the pointers to active
and try V4L2 subdev state, and only then figured out which one to use.

The problem with that approach and a later patch (see Fixes: tag) is that
as sd_state argument to v4l2_subdev_get_try_crop() et al is NULL, there is
now an attempt to dereference that.

Fix this.

Also rewrap lines a little.

Fixes: 0d346d2a6f54 ("media: v4l2-subdev: add subdev-wide state struct")
Cc: stable@vger.kernel.org # for v5.14 and later
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Bingbu Cao <bingbu.cao@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

media: staging/intel-ipu3: Reset imgu_video_device sequence

Reset the sequence number of imgu_video_device on stream start.
Failing to do so results in sequence number getting incremented
for consecutive stream on/off cycles.

Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

media: ipu3-imgu: VIDIOC_QUERYCAP: Fix bus_info

bus_info field had a different value for the media entity and the video
device.

Fixes v4l2-compliance:

v4l2-compliance.cpp(637): media bus_info 'PCI:0000:00:05.0' differs from
V4L2 bus_info 'PCI:viewfinder'

Reviewed-by: Bingbu Cao <bingbu.cao@intel.com>
Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: ipu3-imgu: imgu_fmt: Handle properly try

For a try_fmt call, the node noes not need to be enabled.

Fixes v4l2-compliance

fail: v4l2-test-formats.cpp(717): Video Output Multiplanar is valid, but
no TRY_FMT was implemented
test VIDIOC_TRY_FMT: FAIL

Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: ipu3-imgu: Set valid initial format

The initial format did not have a valid size.

Fixes v4l2-compliance:

fail: v4l2-test-formats.cpp(723): Video Output Multiplanar:
TRY_FMT(G_FMT) != G_FMT
test VIDIOC_TRY_FMT: FAIL

Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/intel-ipu3: Constify static struct v4l2_subdev_internal_ops

The only usage of imgu_subdev_internal_ops is to assign its address to
the internal_ops field in the v4l2_subdev struct, which is a pointer to
const v4l2_subdev_internal_ops. Make it const to allow the compiler to
put it in read-only memory.

Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: v4l2-subdev: add subdev-wide state struct

We have 'struct v4l2_subdev_pad_config' which contains configuration for
a single pad used for the TRY functionality, and an array of those
structs is passed to various v4l2_subdev_pad_ops.

I was working on subdev internal routing between pads, and realized that
there's no way to add TRY functionality for routes, which is not pad
specific configuration. Adding a separate struct for try-route config
wouldn't work either, as e.g. set-fmt needs to know the try-route
configuration to propagate the settings.

This patch adds a new struct, 'struct v4l2_subdev_state' (which at the
moment only contains the v4l2_subdev_pad_config array) and the new
struct is used in most of the places where v4l2_subdev_pad_config was
used. All v4l2_subdev_pad_ops functions taking v4l2_subdev_pad_config
are changed to instead take v4l2_subdev_state.

The changes to drivers/media/v4l2-core/v4l2-subdev.c and
include/media/v4l2-subdev.h were written by hand, and all the driver
changes were done with the semantic patch below. The spatch needs to be
applied to a select list of directories. I used the following shell
commands to apply the spatch:

dirs="drivers/media/i2c drivers/media/platform drivers/media/usb drivers/media/test-drivers/vimc drivers/media/pci drivers/staging/media"
for dir in $dirs; do spatch -j8 --dir --include-headers --no-show-diff --in-place --sp-file v4l2-subdev-state.cocci $dir; done

Note that Coccinelle chokes on a few drivers (gcc extensions?). With
minor changes we can make Coccinelle run fine, and these changes can be
reverted after spatch. The diff for these changes is:

For drivers/media/i2c/s5k5baf.c:

@@ -1481,7 +1481,7 @@ static int s5k5baf_set_selection(struct v4l2_subdev *sd,
&s5k5baf_cis_rect,
v4l2_subdev_get_try_crop(sd, cfg, PAD_CIS),
v4l2_subdev_get_try_compose(sd, cfg, PAD_CIS),
- v4l2_subdev_get_try_crop(sd, cfg, PAD_OUT)
+ v4l2_subdev_get_try_crop(sd, cfg, PAD_OUT),
};
s5k5baf_set_rect_and_adjust(rects, rtype, &sel->r);
return 0;

For drivers/media/platform/s3c-camif/camif-capture.c:

@@ -1230,7 +1230,7 @@ static int s3c_camif_subdev_get_fmt(struct v4l2_subdev *sd,
*mf = camif->mbus_fmt;
break;

- case CAMIF_SD_PAD_SOURCE_C...CAMIF_SD_PAD_SOURCE_P:
+ case CAMIF_SD_PAD_SOURCE_C:
/* crop rectangle at camera interface input */
mf->width = camif->camif_crop.width;
mf->height = camif->camif_crop.height;
@@ -1332,7 +1332,7 @@ static int s3c_camif_subdev_set_fmt(struct v4l2_subdev *sd,
}
break;

- case CAMIF_SD_PAD_SOURCE_C...CAMIF_SD_PAD_SOURCE_P:
+ case CAMIF_SD_PAD_SOURCE_C:
/* Pixel format can be only changed on the sink pad. */
mf->code = camif->mbus_fmt.code;
mf->width = crop->width;

The semantic patch is:

// <smpl>

// Change function parameter

@@
identifier func;
identifier cfg;
@@

func(...,
- struct v4l2_subdev_pad_config *cfg
+ struct v4l2_subdev_state *sd_state
, ...)
{
<...
- cfg
+ sd_state
...>
}

// Change function declaration parameter

@@
identifier func;
identifier cfg;
type T;
@@
T func(...,
- struct v4l2_subdev_pad_config *cfg
+ struct v4l2_subdev_state *sd_state
, ...);

// Change function return value

@@
identifier func;
@@
- struct v4l2_subdev_pad_config
+ struct v4l2_subdev_state
*func(...)
{
...
}

// Change function declaration return value

@@
identifier func;
@@
- struct v4l2_subdev_pad_config
+ struct v4l2_subdev_state
*func(...);

// Some drivers pass a local pad_cfg for a single pad to a called function. Wrap it
// inside a pad_state.

@@
identifier func;
identifier pad_cfg;
@@
func(...)
{
...
struct v4l2_subdev_pad_config pad_cfg;
+ struct v4l2_subdev_state pad_state = { .pads = &pad_cfg };

<+...

(
v4l2_subdev_call
|
sensor_call
|
isi_try_fse
|
isc_try_fse
|
saa_call_all
)
(...,
- &pad_cfg
+ &pad_state
,...)

...+>
}

// If the function uses fields from pad_config, access via state->pads

@@
identifier func;
identifier state;
@@
func(...,
struct v4l2_subdev_state *state
, ...)
{
<...
(
- state->try_fmt
+ state->pads->try_fmt
|
- state->try_crop
+ state->pads->try_crop
|
- state->try_compose
+ state->pads->try_compose
)
...>
}

// If the function accesses the filehandle, use fh->state instead

@@
struct v4l2_subdev_fh *fh;
@@
- fh->pad
+ fh->state

@@
struct v4l2_subdev_fh fh;
@@
- fh.pad
+ fh.state

// Start of vsp1 specific

@@
@@
struct vsp1_entity {
...
- struct v4l2_subdev_pad_config *config;
+ struct v4l2_subdev_state *config;
...
};

@@
symbol entity;
@@
vsp1_entity_init(...)
{
...
entity->config =
- v4l2_subdev_alloc_pad_config
+ v4l2_subdev_alloc_state
(&entity->subdev);
...
}

@@
symbol entity;
@@
vsp1_entity_destroy(...)
{
...
- v4l2_subdev_free_pad_config
+ v4l2_subdev_free_state
(entity->config);
...
}

@exists@
identifier func =~ "(^vsp1.*)|(hsit_set_format)|(sru_enum_frame_size)|(sru_set_format)|(uif_get_selection)|(uif_set_selection)|(uds_enum_frame_size)|(uds_set_format)|(brx_set_format)|(brx_get_selection)|(histo_get_selection)|(histo_set_selection)|(brx_set_selection)";
symbol config;
@@
func(...) {
...
- struct v4l2_subdev_pad_config *config;
+ struct v4l2_subdev_state *config;
...
}

// End of vsp1 specific

// Start of rcar specific

@@
identifier sd;
identifier pad_cfg;
@@
rvin_try_format(...)
{
...
- struct v4l2_subdev_pad_config *pad_cfg;
+ struct v4l2_subdev_state *sd_state;
...
- pad_cfg = v4l2_subdev_alloc_pad_config(sd);
+ sd_state = v4l2_subdev_alloc_state(sd);
<...
- pad_cfg
+ sd_state
...>
- v4l2_subdev_free_pad_config(pad_cfg);
+ v4l2_subdev_free_state(sd_state);
...
}

// End of rcar specific

// Start of rockchip specific

@@
identifier func =~ "(rkisp1_rsz_get_pad_fmt)|(rkisp1_rsz_get_pad_crop)|(rkisp1_rsz_register)";
symbol rsz;
symbol pad_cfg;
@@

func(...)
{
+ struct v4l2_subdev_state state = { .pads = rsz->pad_cfg };
...
- rsz->pad_cfg
+ &state
...
}

@@
identifier func =~ "(rkisp1_isp_get_pad_fmt)|(rkisp1_isp_get_pad_crop)";
symbol isp;
symbol pad_cfg;
@@

func(...)
{
+ struct v4l2_subdev_state state = { .pads = isp->pad_cfg };
...
- isp->pad_cfg
+ &state
...
}

@@
symbol rkisp1;
symbol isp;
symbol pad_cfg;
@@

rkisp1_isp_register(...)
{
+ struct v4l2_subdev_state state = { .pads = rkisp1->isp.pad_cfg };
...
- rkisp1->isp.pad_cfg
+ &state
...
}

// End of rockchip specific

// Start of tegra-video specific

@@
identifier sd;
identifier pad_cfg;
@@
__tegra_channel_try_format(...)
{
...
- struct v4l2_subdev_pad_config *pad_cfg;
+ struct v4l2_subdev_state *sd_state;
...
- pad_cfg = v4l2_subdev_alloc_pad_config(sd);
+ sd_state = v4l2_subdev_alloc_state(sd);
<...
- pad_cfg
+ sd_state
...>
- v4l2_subdev_free_pad_config(pad_cfg);
+ v4l2_subdev_free_state(sd_state);
...
}

@@
identifier sd_state;
@@
__tegra_channel_try_format(...)
{
...
struct v4l2_subdev_state *sd_state;
<...
- sd_state->try_crop
+ sd_state->pads->try_crop
...>
}

// End of tegra-video specific

// </smpl>

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/intel-ipu3: Fix race condition during set_fmt

Do not modify imgu_pipe->nodes[inode].vdev_fmt.fmt.pix_mp, until the
format has been correctly validated.

Otherwise, even if we use a backup variable, there is a period of time
where imgu_pipe->nodes[inode].vdev_fmt.fmt.pix_mp might have an invalid
value that can be used by other functions.

Cc: stable@vger.kernel.org
Fixes: ad91849996f9 ("media: staging/intel-ipu3: Fix set_fmt error handling")
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/intel-ipu3: Fix memory leak in imu_fmt

We are losing the reference to an allocated memory if try. Change the
order of the check to avoid that.

Cc: stable@vger.kernel.org
Fixes: 6d5f26f2e045 ("media: staging/intel-ipu3-v4l: reduce kernel stack usage")
Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/intel-ipu3: Fix set_fmt error handling

If there in an error during a set_fmt, do not overwrite the previous
sizes with the invalid config.

Without this patch, v4l2-compliance ends up allocating 4GiB of RAM and
causing the following OOPs

[ 38.662975] ipu3-imgu 0000:00:05.0: swiotlb buffer is full (sz: 4096 bytes)
[ 38.662980] DMA: Out of SW-IOMMU space for 4096 bytes at device 0000:00:05.0
[ 38.663010] general protection fault: 0000 [#1] PREEMPT SMP

Cc: stable@vger.kernel.org
Fixes: 6d5f26f2e045 ("media: staging/intel-ipu3-v4l: reduce kernel stack usage")
Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/intel-ipu3: Do not zero reserved fields

Core code already clears reserved fields of struct v4l2_pix_format_mplane,
check commit 4e1e0eb0e074 ("media: v4l2-ioctl: Zero v4l2_plane_pix_format
reserved fields").

Signed-off-by: Ricardo Ribalda <ribalda@chromium.org>
Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/intel-ipu3: Make use of V4L2_CAP_IO_MC

Set the V4L2_CAP_IO_MC capability flag and remove the driver specific
vidioc_enum_{input,output}, vidioc_g_{input,output} and
vidioc_s_{input,output} callbacks.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/intel-ipu3: Simplify single goto jump

On successful node setup, the code jumps to a cleanup label to perform
nodes cleanup. This only call to cleanup using goto label can be
included in the for / if blocks to make it look more associated.

Signed-off-by: Deepak R Varma <mh12gx2825@gmail.com>
Reviewed-by: Stefano Brivio <sbrivio@redhat.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/intel-ipu3: Remove extra blank lines

Remove extra blank lines from the code blocks.

Signed-off-by: Deepak R Varma <mh12gx2825@gmail.com>
Reviewed-by: Stefano Brivio <sbrivio@redhat.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/intel-ipu3: Implement lock for stream on/off operations

Currently concurrent stream off operations on ImgU nodes are not
synchronized, leading to use-after-free bugs (as reported by KASAN).

[ 250.090724] BUG: KASAN: use-after-free in
ipu3_dmamap_free+0xc5/0x116 [ipu3_imgu]
[ 250.090726] Read of size 8 at addr ffff888127b29bc0 by task
yavta/18836
[ 250.090731] Hardware name: HP Soraka/Soraka, BIOS
Google_Soraka.10431.17.0 03/22/2018
[ 250.090732] Call Trace:
[ 250.090735] dump_stack+0x6a/0xb1
[ 250.090739] print_address_description+0x8e/0x279
[ 250.090743] ? ipu3_dmamap_free+0xc5/0x116 [ipu3_imgu]
[ 250.090746] kasan_report+0x260/0x28a
[ 250.090750] ipu3_dmamap_free+0xc5/0x116 [ipu3_imgu]
[ 250.090754] ipu3_css_pool_cleanup+0x24/0x37 [ipu3_imgu]
[ 250.090759] ipu3_css_pipeline_cleanup+0x61/0xb9 [ipu3_imgu]
[ 250.090763] ipu3_css_stop_streaming+0x1f2/0x321 [ipu3_imgu]
[ 250.090768] imgu_s_stream+0x94/0x443 [ipu3_imgu]
[ 250.090772] ? ipu3_vb2_buf_queue+0x280/0x280 [ipu3_imgu]
[ 250.090775] ? vb2_dma_sg_unmap_dmabuf+0x16/0x6f [videobuf2_dma_sg]
[ 250.090778] ? vb2_buffer_in_use+0x36/0x58 [videobuf2_common]
[ 250.090782] ipu3_vb2_stop_streaming+0xf9/0x135 [ipu3_imgu]

Implemented a lock to synchronize imgu stream on / off operations and
the modification of streaming flag (in struct imgu_device), to prevent
these issues.

Reported-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Suggested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Rajmohan Mani <rajmohan.mani@intel.com>
Signed-off-by: Bingbu Cao <bingbu.cao@intel.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/media: rename VFL_TYPE_GRABBER to _VIDEO

'GRABBER' is a weird name, all other types map to the /dev
device names. Rename to 'VIDEO' to be consistent with the
other types.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: Revert "media: staging/intel-ipu3: make imgu use fixed running mode"

This reverts commit e878742c83ec26ef256ebb6b36a4d44644548f25.

Imgu should still keep the capability and flexibility to allow user to
run 2 video pipes, as the user may use the video pipe to capture still
frames with less system load and power than still pipe.

Suggested-by: Tomasz Figa <tfiga@chromium.org>
Signed-off-by: Bingbu Cao <bingbu.cao@intel.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/intel-ipu3: make imgu use fixed running mode

Currently, the imgu running mode need user to set by v4l2 ctrl.
However, imgu only support 2 pipes and 2 operation modes - video and
still. This patch make the first imgu subdev running as video and second
one running as still, it will make the user understand easily, it can
also cover current camera use cases requirement. The running mode is set
during subdev registering, no race-condition after change, so it is safe
to change the mode data type to integer.

Signed-off-by: Bingbu Cao <bingbu.cao@intel.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/intel-ipu3: set the main output as mandatory

From firmware perspective, the main output for each pipe is mandatory
and must be enabled. This patch set the link between imgu subdev and
main output as IMMUTABLE by default.

Signed-off-by: Bingbu Cao <bingbu.cao@intel.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: v4l2: Get rid of ->vidioc_enum_fmt_vid_{cap, out}_mplane

Support for multiplanar and singleplanar formats is mutually exclusive,
at least in practice. In our attempt to unify support for support for
mplane and !mplane in v4l, let's get rid of the
->vidioc_enum_fmt_{vid,out}_cap_mplane() hooks and call
->vidioc_enum_fmt_{vid,out}_cap() instead.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
[hverkuil-cisco@xs4all.nl: fix typos: pirv -> priv and prov -> priv]
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: ipu3-imgu: Use %u for formatting unsigned values (not %d)

The driver used %d for formatting unsigned values. Use %u instead.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Bingbu Cao <bingbu.cao@intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/intel-ipu3: parameter buffer refactoring

Current ImgU driver processes and releases the parameter buffer
immediately after queued from user. This does not align with other
image buffers which are grouped in sets and used for the same frame.
If user queues multiple parameter buffers continuously, only the last
one will take effect.
To make consistent buffers usage, this patch changes the parameter
buffer handling and group parameter buffer with other image buffers
for each frame.
Each time driver will queue one more group of buffers when previous
frame processed and buffers consumed by css.

Signed-off-by: Tianshu Qiu <tian.shu.qiu@intel.com>
Signed-off-by: Bingbu Cao <bingbu.cao@intel.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/intel-ipu3-v4l: reduce kernel stack usage

The v4l2_pix_format_mplane structure is too large to be put on the kernel
stack, as we can see in 32-bit builds:

drivers/staging/media/ipu3/ipu3-v4l2.c: In function 'imgu_fmt':
drivers/staging/media/ipu3/ipu3-v4l2.c:753:1: error: the frame size of 1028 bytes is larger than 1024 bytes [-Werror=frame-larger-than=]

By dynamically allocating this array, the stack usage goes down to an
acceptable 272 bytes for the same x86-32 configuration.

Fixes: a0ca1627b450 ("media: staging/intel-ipu3: Add v4l2 driver based on media framework")

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: ipu3-imgu: Prefix functions with imgu_* instead of ipu3_*

This addresses the below TODO item, no function related changes:

- Prefix imgu for all public APIs, i.e. change ipu3_v4l2_register() to
imgu_v4l2_register(). (Sakari)

The changes were obtained by applying the following perl script
to driver code under drivers/staging/media/ipu3/.

perl -pi.back -e 's/ipu3_(?!uapi)/imgu_/g;'

Signed-off-by: Yong Zhi <yong.zhi@intel.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: ipu3-imgu: Use MENU type for mode control

This addresses the below TODO item.
- Use V4L2_CTRL_TYPE_MENU for dual-pipe mode control. (Sakari)

Signed-off-by: Yong Zhi <yong.zhi@intel.com>
Reviewed-by: Tomasz Figa <tfiga@chromium.org>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/intel-ipu3: Add dual pipe support

This patch adds support to run dual pipes simultaneously.
A private ioctl to configure the pipe mode (video or still)
is also implemented.

IPU3 hardware supports a maximum of 2 streams per pipe.
With the support of dual pipes, more than 2 stream outputs
can be achieved.

This helps to support advanced camera features like
Continuous View Finder (CVF) and Snapshot During Video(SDV).

Extend ipu3 IMGU driver to support dual pipes

1. Extend current IMGU device to contain 2 groups
of video nodes and 2 subdevs
2. Extend current css to support 2 pipeline and make
CSS APIs to support 2 pipe
3. Add a v4l2 ctrl to allow user to specify the mode
of the pipe
4. Check media pipeline link status to get enabled
pipes

Signed-off-by: Bingbu Cao <bingbu.cao@intel.com>
Signed-off-by: Tian Shu Qiu <tian.shu.qiu@intel.com>
Signed-off-by: Yong Zhi <yong.zhi@intel.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/intel-ipu3: Add v4l2 driver based on media framework

Implement video driver that utilizes v4l2, vb2 queue support
and media controller APIs. The driver exposes single subdevice and
six nodes.

Signed-off-by: Yong Zhi <yong.zhi@intel.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>