media: v4l2-subdev: Rename .init_cfg() operation to .init_state()

The subdev .init_cfg() operation is affected by two issues:

- It has long been extended to initialize a whole v4l2_subdev_state
instead of just a v4l2_subdev_pad_config, but its name has stuck
around.

- Despite operating on a whole subdev state and not being directly
exposed to the subdev users (either in-kernel or through the userspace
API), .init_cfg() is categorized as a subdev pad operation.

This participates in making the subdev API confusing for new developers.
Fix it by renaming the operation to .init_state(), and make it a subdev
internal operation.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Michael Riesch <michael.riesch@wolfvision.net> # for imx415
Acked-by: Shuah Khan <skhan@linuxfoundation.org> # for vimc
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
[Sakari Ailus: Resolved a conflict in Renesas vsp1 driver.]
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

media: v4l: subdev: Switch to stream-aware state functions

Switch all drivers accessing sub-device state to use the stream-aware
functions. We will soon remove the old ones.

This patch has been generated using the following Coccinelle script:

---------8<------------
@@
expression E1, E2, E3;

@@

- v4l2_subdev_get_pad_format(E1, E2, E3)
+ v4l2_subdev_state_get_format(E2, E3)

@@
expression E1, E2, E3;

@@

- v4l2_subdev_get_pad_crop(E1, E2, E3)
+ v4l2_subdev_state_get_crop(E2, E3)

@@
expression E1, E2, E3;

@@

- v4l2_subdev_get_pad_compose(E1, E2, E3)
+ v4l2_subdev_state_get_compose(E2, E3)

@@
expression E1, E2, E3;

@@

- v4l2_subdev_get_try_format(E1, E2, E3)
+ v4l2_subdev_state_get_format(E2, E3)

@@
expression E1, E2, E3;

@@

- v4l2_subdev_get_try_crop(E1, E2, E3)
+ v4l2_subdev_state_get_crop(E2, E3)

@@
expression E1, E2, E3;

@@

- v4l2_subdev_get_try_compose(E1, E2, E3)
+ v4l2_subdev_state_get_compose(E2, E3)
---------8<------------

Additionally drivers/media/i2c/s5k5baf.c and
drivers/media/platform/samsung/s3c-camif/camif-capture.c have been
manually changed as Coccinelle didn't. Further local variables have been
removed as they became unused as a result of the other changes.

Also Coccinelle introduced indentation by space in files
drivers/media/i2c/st-mipid02.c and
drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c. This has been also
corrected.

The diff from Coccinelle-generated changes are:

> diff --git b/drivers/media/i2c/imx319.c a/drivers/media/i2c/imx319.c
> index e549692ff478..420984382173 100644
> --- b/drivers/media/i2c/imx319.c
> +++ a/drivers/media/i2c/imx319.c
> @@ -2001,7 +2001,6 @@ static int imx319_do_get_pad_format(struct imx319 *imx319,
> struct v4l2_subdev_format *fmt)
> {
> struct v4l2_mbus_framefmt *framefmt;
> - struct v4l2_subdev *sd = &imx319->sd;
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> diff --git b/drivers/media/i2c/imx355.c a/drivers/media/i2c/imx355.c
> index 96bdde685d65..e1b1d2fc79dd 100644
> --- b/drivers/media/i2c/imx355.c
> +++ a/drivers/media/i2c/imx355.c
> @@ -1299,7 +1299,6 @@ static int imx355_do_get_pad_format(struct imx355 *imx355,
> struct v4l2_subdev_format *fmt)
> {
> struct v4l2_mbus_framefmt *framefmt;
> - struct v4l2_subdev *sd = &imx355->sd;
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> diff --git b/drivers/media/i2c/ov08x40.c a/drivers/media/i2c/ov08x40.c
> index ca799bbcfdb7..abbb0b774d43 100644
> --- b/drivers/media/i2c/ov08x40.c
> +++ a/drivers/media/i2c/ov08x40.c
> @@ -2774,7 +2774,6 @@ static int ov08x40_do_get_pad_format(struct ov08x40 *ov08x,
> struct v4l2_subdev_format *fmt)
> {
> struct v4l2_mbus_framefmt *framefmt;
> - struct v4l2_subdev *sd = &ov08x->sd;
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> diff --git b/drivers/media/i2c/ov13858.c a/drivers/media/i2c/ov13858.c
> index 7816d9787c61..09387e335d80 100644
> --- b/drivers/media/i2c/ov13858.c
> +++ a/drivers/media/i2c/ov13858.c
> @@ -1316,7 +1316,6 @@ static int ov13858_do_get_pad_format(struct ov13858 *ov13858,
> struct v4l2_subdev_format *fmt)
> {
> struct v4l2_mbus_framefmt *framefmt;
> - struct v4l2_subdev *sd = &ov13858->sd;
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> diff --git b/drivers/media/i2c/ov13b10.c a/drivers/media/i2c/ov13b10.c
> index 268cd4b03f9c..c06411d5ee2b 100644
> --- b/drivers/media/i2c/ov13b10.c
> +++ a/drivers/media/i2c/ov13b10.c
> @@ -1001,7 +1001,6 @@ static int ov13b10_do_get_pad_format(struct ov13b10 *ov13b,
> struct v4l2_subdev_format *fmt)
> {
> struct v4l2_mbus_framefmt *framefmt;
> - struct v4l2_subdev *sd = &ov13b->sd;
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> diff --git b/drivers/media/i2c/s5c73m3/s5c73m3-core.c a/drivers/media/i2c/s5c73m3/s5c73m3-core.c
> index 47605e36bc60..8f9b5713daf7 100644
> --- b/drivers/media/i2c/s5c73m3/s5c73m3-core.c
> +++ a/drivers/media/i2c/s5c73m3/s5c73m3-core.c
> @@ -819,7 +819,6 @@ static void s5c73m3_oif_try_format(struct s5c73m3 *state,
> struct v4l2_subdev_format *fmt,
> const struct s5c73m3_frame_size **fs)
> {
> - struct v4l2_subdev *sd = &state->sensor_sd;
> u32 code;
>
> switch (fmt->pad) {
> diff --git a/drivers/media/i2c/s5k5baf.c b/drivers/media/i2c/s5k5baf.c
> index 67da2045f543..03ccfb0e1e11 100644
> --- a/drivers/media/i2c/s5k5baf.c
> +++ b/drivers/media/i2c/s5k5baf.c
> @@ -1472,14 +1472,11 @@ static int s5k5baf_set_selection(struct v4l2_subdev *sd,
>
> if (sel->which == V4L2_SUBDEV_FORMAT_TRY) {
> rects = (struct v4l2_rect * []) {
> - &s5k5baf_cis_rect,
> - v4l2_subdev_get_try_crop(sd, sd_state,
> - PAD_CIS),
> - v4l2_subdev_get_try_compose(sd, sd_state,
> - PAD_CIS),
> - v4l2_subdev_get_try_crop(sd, sd_state,
> - PAD_OUT)
> - };
> + &s5k5baf_cis_rect,
> + v4l2_subdev_state_get_crop(sd_state, PAD_CIS),
> + v4l2_subdev_state_get_compose(sd_state, PAD_CIS),
> + v4l2_subdev_state_get_crop(sd_state, PAD_OUT)
> + };
> s5k5baf_set_rect_and_adjust(rects, rtype, &sel->r);
> return 0;
> }
> diff --git b/drivers/media/platform/samsung/s3c-camif/camif-capture.c a/drivers/media/platform/samsung/s3c-camif/camif-capture.c
> index 295e083f38e8..be58260ea67e 100644
> --- b/drivers/media/platform/samsung/s3c-camif/camif-capture.c
> +++ a/drivers/media/platform/samsung/s3c-camif/camif-capture.c
> @@ -1216,7 +1216,7 @@ static int s3c_camif_subdev_get_fmt(struct v4l2_subdev *sd,
> struct v4l2_mbus_framefmt *mf = &fmt->format;
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> - mf = v4l2_subdev_get_try_format(sd, sd_state, fmt->pad);
> + mf = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> fmt->format = *mf;
> return 0;
> }
> @@ -1305,7 +1305,7 @@ static int s3c_camif_subdev_set_fmt(struct v4l2_subdev *sd,
> __camif_subdev_try_format(camif, mf, fmt->pad);
>
> if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> - mf = v4l2_subdev_get_try_format(sd, sd_state, fmt->pad);
> + mf = v4l2_subdev_state_get_format(sd_state, fmt->pad);
> *mf = fmt->format;
> mutex_unlock(&camif->lock);
> return 0;
> diff --git b/drivers/media/platform/ti/cal/cal-camerarx.c a/drivers/media/platform/ti/cal/cal-camerarx.c
> index cea454ed9c20..61433744c6c4 100644
> --- b/drivers/media/platform/ti/cal/cal-camerarx.c
> +++ a/drivers/media/platform/ti/cal/cal-camerarx.c
> @@ -621,8 +621,6 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
> struct v4l2_subdev_state *state,
> struct v4l2_subdev_mbus_code_enum *code)
> {
> - struct cal_camerarx *phy = to_cal_camerarx(sd);
> -
> /* No transcoding, source and sink codes must match. */
> if (cal_rx_pad_is_source(code->pad)) {
> struct v4l2_mbus_framefmt *fmt;
> diff --git b/drivers/staging/media/imx/imx-ic-prp.c a/drivers/staging/media/imx/imx-ic-prp.c
> index dd558fac6477..61d69f19657e 100644
> --- b/drivers/staging/media/imx/imx-ic-prp.c
> +++ a/drivers/staging/media/imx/imx-ic-prp.c
> @@ -82,8 +82,6 @@ static struct v4l2_mbus_framefmt *
> __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_state *sd_state,
> unsigned int pad, enum v4l2_subdev_format_whence which)
> {
> - struct imx_ic_priv *ic_priv = priv->ic_priv;
> -
> if (which == V4L2_SUBDEV_FORMAT_TRY)
> return v4l2_subdev_state_get_format(sd_state, pad);
> else
> diff --git b/drivers/staging/media/imx/imx-ic-prpencvf.c a/drivers/staging/media/imx/imx-ic-prpencvf.c
> index 02db7dbb884b..ec73c901079e 100644
> --- b/drivers/staging/media/imx/imx-ic-prpencvf.c
> +++ a/drivers/staging/media/imx/imx-ic-prpencvf.c
> @@ -790,8 +790,6 @@ static struct v4l2_mbus_framefmt *
> __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_state *sd_state,
> unsigned int pad, enum v4l2_subdev_format_whence which)
> {
> - struct imx_ic_priv *ic_priv = priv->ic_priv;
> -
> if (which == V4L2_SUBDEV_FORMAT_TRY)
> return v4l2_subdev_state_get_format(sd_state, pad);
> else
> diff --git a/drivers/media/i2c/st-mipid02.c b/drivers/media/i2c/st-mipid02.c
> index 9c9361354c00..b08a249b5fdd 100644
> --- a/drivers/media/i2c/st-mipid02.c
> +++ b/drivers/media/i2c/st-mipid02.c
> @@ -751,7 +751,7 @@ static void mipid02_set_fmt_source(struct v4l2_subdev *sd,
> format->format = bridge->fmt;
> else
> format->format = *v4l2_subdev_state_get_format(sd_state,
> - MIPID02_SINK_0);
> + MIPID02_SINK_0);
>
> /* but code may need to be converted */
> format->format.code = serial_to_parallel_code(format->format.code);
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
> index 117912d3bfbd..96353648c032 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
> @@ -319,7 +319,7 @@ static void rkisp1_isp_start(struct rkisp1_isp *isp,
> rkisp1_write(rkisp1, RKISP1_CIF_ISP_CTRL, val);
>
> src_fmt = v4l2_subdev_state_get_format(sd_state,
> - RKISP1_ISP_PAD_SOURCE_VIDEO);
> + RKISP1_ISP_PAD_SOURCE_VIDEO);
> src_info = rkisp1_mbus_info_get_by_code(src_fmt->code);
>
> if (src_info->pixel_enc != V4L2_PIXEL_ENC_BAYER)
> @@ -475,9 +475,9 @@ static void rkisp1_isp_set_src_fmt(struct rkisp1_isp *isp,
> sink_fmt = v4l2_subdev_state_get_format(sd_state,
> RKISP1_ISP_PAD_SINK_VIDEO);
> src_fmt = v4l2_subdev_state_get_format(sd_state,
> - RKISP1_ISP_PAD_SOURCE_VIDEO);
> + RKISP1_ISP_PAD_SOURCE_VIDEO);
> src_crop = v4l2_subdev_state_get_crop(sd_state,
> - RKISP1_ISP_PAD_SOURCE_VIDEO);
> + RKISP1_ISP_PAD_SOURCE_VIDEO);
>
> /*
> * Media bus code. The ISP can operate in pass-through mode (Bayer in,

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

media: imx: utils: Enclose IMX_BUS_FMTS macro in parenthesis

Conform to kernel coding style by wrapping macro in parenthesis. Issue
found by checkpatch.

Signed-off-by: Angel Alberto Carretero <angelalbertoc.r@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

media: Prefer designated initializers over memset for subdev pad ops

Structures passed to subdev pad operations are all zero-initialized, but
not always with the same kind of code constructs. While most drivers
used designated initializers, which zero all the fields that are not
specified, when declaring variables, some use memset(). Those two
methods lead to the same end result, and, depending on compiler
optimizations, may even be completely equivalent, but they're not
consistent.

Improve coding style consistency by using designated initializers
instead of calling memset(). Where applicable, also move the variables
to inner scopes of for loops to ensure correct initialization in all
iterations.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Lad Prabhakar <prabhakar.csengg@gmail.com> # For am437x
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham+renesas@ideasonboard.com>
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

staging: media: imx: Drop unused helper functions

The imx_media_find_subdev_by_fwnode(),
imx_media_find_subdev_by_devname(), imx_media_pipeline_csi2_channel()
and imx_media_pipeline_video_device() helper functions are not used.
Drop them.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

media: imx: Use get_mbus_config instead of parsing upstream DT endpoints

Stop parsing upstream neighbors' device-tree endpoints to retrieve the
media bus configuration. Instead use the get_mbus_config op and throw an
error if the upstream subdevice does not implement it.

Also drop the corresponding TODO entry and the now unused
imx_media_get_pad_fwnode() function.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Tested-by: Marco Felsch <m.felsch@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

media: mc: convert pipeline funcs to take media_pad

Now that the pipeline is stored into pads instead of entities, we can
change the relevant functions to take pads instead of entities.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

media: mc: entity: Add media_entity_pipeline() to access the media pipeline

Replace direct access to the pipe field in drivers with a new helper
function. This will allow easier refactoring of media pipeline handling
in the MC core behind the scenes without affecting drivers.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

media: mc-entity: Rename media_entity_remote_pad() to media_pad_remote_pad_first()

The media_entity_remote_pad() is misnamed, as it operates on a pad and
not an entity. Rename it to media_pad_remote_pad_first() to clarify its
behaviour.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

media: imx: Remove unused functions

Neither imx_media_mbus_fmt_to_ipu_image nor imx_media_ipu_image_to_mbus_fmt
were used anywhere.

Signed-off-by: Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: v4l2-subdev: add subdev-wide state struct

We have 'struct v4l2_subdev_pad_config' which contains configuration for
a single pad used for the TRY functionality, and an array of those
structs is passed to various v4l2_subdev_pad_ops.

I was working on subdev internal routing between pads, and realized that
there's no way to add TRY functionality for routes, which is not pad
specific configuration. Adding a separate struct for try-route config
wouldn't work either, as e.g. set-fmt needs to know the try-route
configuration to propagate the settings.

This patch adds a new struct, 'struct v4l2_subdev_state' (which at the
moment only contains the v4l2_subdev_pad_config array) and the new
struct is used in most of the places where v4l2_subdev_pad_config was
used. All v4l2_subdev_pad_ops functions taking v4l2_subdev_pad_config
are changed to instead take v4l2_subdev_state.

The changes to drivers/media/v4l2-core/v4l2-subdev.c and
include/media/v4l2-subdev.h were written by hand, and all the driver
changes were done with the semantic patch below. The spatch needs to be
applied to a select list of directories. I used the following shell
commands to apply the spatch:

dirs="drivers/media/i2c drivers/media/platform drivers/media/usb drivers/media/test-drivers/vimc drivers/media/pci drivers/staging/media"
for dir in $dirs; do spatch -j8 --dir --include-headers --no-show-diff --in-place --sp-file v4l2-subdev-state.cocci $dir; done

Note that Coccinelle chokes on a few drivers (gcc extensions?). With
minor changes we can make Coccinelle run fine, and these changes can be
reverted after spatch. The diff for these changes is:

For drivers/media/i2c/s5k5baf.c:

@@ -1481,7 +1481,7 @@ static int s5k5baf_set_selection(struct v4l2_subdev *sd,
&s5k5baf_cis_rect,
v4l2_subdev_get_try_crop(sd, cfg, PAD_CIS),
v4l2_subdev_get_try_compose(sd, cfg, PAD_CIS),
- v4l2_subdev_get_try_crop(sd, cfg, PAD_OUT)
+ v4l2_subdev_get_try_crop(sd, cfg, PAD_OUT),
};
s5k5baf_set_rect_and_adjust(rects, rtype, &sel->r);
return 0;

For drivers/media/platform/s3c-camif/camif-capture.c:

@@ -1230,7 +1230,7 @@ static int s3c_camif_subdev_get_fmt(struct v4l2_subdev *sd,
*mf = camif->mbus_fmt;
break;

- case CAMIF_SD_PAD_SOURCE_C...CAMIF_SD_PAD_SOURCE_P:
+ case CAMIF_SD_PAD_SOURCE_C:
/* crop rectangle at camera interface input */
mf->width = camif->camif_crop.width;
mf->height = camif->camif_crop.height;
@@ -1332,7 +1332,7 @@ static int s3c_camif_subdev_set_fmt(struct v4l2_subdev *sd,
}
break;

- case CAMIF_SD_PAD_SOURCE_C...CAMIF_SD_PAD_SOURCE_P:
+ case CAMIF_SD_PAD_SOURCE_C:
/* Pixel format can be only changed on the sink pad. */
mf->code = camif->mbus_fmt.code;
mf->width = crop->width;

The semantic patch is:

// <smpl>

// Change function parameter

@@
identifier func;
identifier cfg;
@@

func(...,
- struct v4l2_subdev_pad_config *cfg
+ struct v4l2_subdev_state *sd_state
, ...)
{
<...
- cfg
+ sd_state
...>
}

// Change function declaration parameter

@@
identifier func;
identifier cfg;
type T;
@@
T func(...,
- struct v4l2_subdev_pad_config *cfg
+ struct v4l2_subdev_state *sd_state
, ...);

// Change function return value

@@
identifier func;
@@
- struct v4l2_subdev_pad_config
+ struct v4l2_subdev_state
*func(...)
{
...
}

// Change function declaration return value

@@
identifier func;
@@
- struct v4l2_subdev_pad_config
+ struct v4l2_subdev_state
*func(...);

// Some drivers pass a local pad_cfg for a single pad to a called function. Wrap it
// inside a pad_state.

@@
identifier func;
identifier pad_cfg;
@@
func(...)
{
...
struct v4l2_subdev_pad_config pad_cfg;
+ struct v4l2_subdev_state pad_state = { .pads = &pad_cfg };

<+...

(
v4l2_subdev_call
|
sensor_call
|
isi_try_fse
|
isc_try_fse
|
saa_call_all
)
(...,
- &pad_cfg
+ &pad_state
,...)

...+>
}

// If the function uses fields from pad_config, access via state->pads

@@
identifier func;
identifier state;
@@
func(...,
struct v4l2_subdev_state *state
, ...)
{
<...
(
- state->try_fmt
+ state->pads->try_fmt
|
- state->try_crop
+ state->pads->try_crop
|
- state->try_compose
+ state->pads->try_compose
)
...>
}

// If the function accesses the filehandle, use fh->state instead

@@
struct v4l2_subdev_fh *fh;
@@
- fh->pad
+ fh->state

@@
struct v4l2_subdev_fh fh;
@@
- fh.pad
+ fh.state

// Start of vsp1 specific

@@
@@
struct vsp1_entity {
...
- struct v4l2_subdev_pad_config *config;
+ struct v4l2_subdev_state *config;
...
};

@@
symbol entity;
@@
vsp1_entity_init(...)
{
...
entity->config =
- v4l2_subdev_alloc_pad_config
+ v4l2_subdev_alloc_state
(&entity->subdev);
...
}

@@
symbol entity;
@@
vsp1_entity_destroy(...)
{
...
- v4l2_subdev_free_pad_config
+ v4l2_subdev_free_state
(entity->config);
...
}

@exists@
identifier func =~ "(^vsp1.*)|(hsit_set_format)|(sru_enum_frame_size)|(sru_set_format)|(uif_get_selection)|(uif_set_selection)|(uds_enum_frame_size)|(uds_set_format)|(brx_set_format)|(brx_get_selection)|(histo_get_selection)|(histo_set_selection)|(brx_set_selection)";
symbol config;
@@
func(...) {
...
- struct v4l2_subdev_pad_config *config;
+ struct v4l2_subdev_state *config;
...
}

// End of vsp1 specific

// Start of rcar specific

@@
identifier sd;
identifier pad_cfg;
@@
rvin_try_format(...)
{
...
- struct v4l2_subdev_pad_config *pad_cfg;
+ struct v4l2_subdev_state *sd_state;
...
- pad_cfg = v4l2_subdev_alloc_pad_config(sd);
+ sd_state = v4l2_subdev_alloc_state(sd);
<...
- pad_cfg
+ sd_state
...>
- v4l2_subdev_free_pad_config(pad_cfg);
+ v4l2_subdev_free_state(sd_state);
...
}

// End of rcar specific

// Start of rockchip specific

@@
identifier func =~ "(rkisp1_rsz_get_pad_fmt)|(rkisp1_rsz_get_pad_crop)|(rkisp1_rsz_register)";
symbol rsz;
symbol pad_cfg;
@@

func(...)
{
+ struct v4l2_subdev_state state = { .pads = rsz->pad_cfg };
...
- rsz->pad_cfg
+ &state
...
}

@@
identifier func =~ "(rkisp1_isp_get_pad_fmt)|(rkisp1_isp_get_pad_crop)";
symbol isp;
symbol pad_cfg;
@@

func(...)
{
+ struct v4l2_subdev_state state = { .pads = isp->pad_cfg };
...
- isp->pad_cfg
+ &state
...
}

@@
symbol rkisp1;
symbol isp;
symbol pad_cfg;
@@

rkisp1_isp_register(...)
{
+ struct v4l2_subdev_state state = { .pads = rkisp1->isp.pad_cfg };
...
- rkisp1->isp.pad_cfg
+ &state
...
}

// End of rockchip specific

// Start of tegra-video specific

@@
identifier sd;
identifier pad_cfg;
@@
__tegra_channel_try_format(...)
{
...
- struct v4l2_subdev_pad_config *pad_cfg;
+ struct v4l2_subdev_state *sd_state;
...
- pad_cfg = v4l2_subdev_alloc_pad_config(sd);
+ sd_state = v4l2_subdev_alloc_state(sd);
<...
- pad_cfg
+ sd_state
...>
- v4l2_subdev_free_pad_config(pad_cfg);
+ v4l2_subdev_free_state(sd_state);
...
}

@@
identifier sd_state;
@@
__tegra_channel_try_format(...)
{
...
struct v4l2_subdev_state *sd_state;
<...
- sd_state->try_crop
+ sd_state->pads->try_crop
...>
}

// End of tegra-video specific

// </smpl>

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Add ability to filter pixel formats by mbus code

Add a media bus code argument to the imx_media_enum_pixel_formats(). If
set to a non-zero value, the function will only consider pixel formats
that match the given media bus code.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: csi: Lookup upstream endpoint with imx_media_get_pad_fwnode

Fix the 1:1 port-id:pad-index assumption for the upstream subdevice, by
searching the upstream subdevice's endpoints for one that maps to the
pad's index. This is carried out by a new reverse mapping function
imx_media_get_pad_fwnode().

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Default colorspace to SRGB

The function imx_media_init_mbus_fmt() initializes the imx subdevice
mbus colorimetry to some sane defaults when the subdevice is registered.
Currently it guesses at a colorspace based on the passed mbus pixel
format. If the format is RGB, it chooses colorspace V4L2_COLORSPACE_SRGB,
and if the format is YUV, it chooses V4L2_COLORSPACE_SMPTE170M.

While that might be a good guess, it's not necessarily true that a RGB
pixel format encoding uses a SRGB colorspace, or that a YUV encoding
uses a SMPTE170M colorspace. Instead of making this dubious guess,
just default the colorspace to SRGB.

Reported-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Constify some mbus and ipu_image arguments

The imx_media_mbus_fmt_to_pix_fmt() and imx_media_mbus_fmt_to_ipu_image()
functions do not need to modify their mbus argument, and
imx_media_ipu_image_to_mbus_fmt() does not need to modify its ipu_image
argument. Make them const.

[slongerbeam@gmail.com: Constified mbus arg to
imx_media_mbus_fmt_to_ipu_image(), and ipu_image
arg to imx_media_ipu_image_to_mbus_fmt(), as well]

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Rename format lookup and enumeration functions

Rename the format lookup and enumeration functions according to their
usage:

- Rename imx_media_(find|enum)_format() to *_pixel_format() to
explicitly state on what formats the functions operate. This aligns
the naming scheme with the media bus and IPU format functions that
already end with *_mbus_format() and *_ipu_formats().

- Rename all enumeration functions to pluralize 'formats' at the end, as
they enumerate multiple formats.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Split find|enum_format into fourcc and mbus functions

To make the code easier to follow, split up find_format() into separate
search functions for pixel formats and media-bus codes, and inline
find_format() into the exported functions imx_media_find_format()
and imx_media_find_mbus_format().

Do the equivalent for enum_format().

Also add comment blocks for the exported find|enum functions.

The convenience functions imx_media_find_ipu_format() and
imx_media_enum_ipu_format() can now be made inline and moved to
imx-media.h.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Make imx_media_pixfmt handle variable number of codes

The imx_media_pixfmt structures include a codes member that stores
media bus codes as a fixed array of 4 integers. The functions dealing
with the imx_media_pixfmt structures assume that the array of codes is
terminated by a 0 element. This mechanism is fragile, as demonstrated
by several instances of the structure containing 4 non-zero codes.

Fix this by turning the array into a pointer, and providing an
IMX_BUS_FMTS macro to initialize the codes member with a guaranteed 0
element at the end.

[Fixed a NULL deref of the codes pointer in a couple places]
[Added more comments for the struct imx_media_pixfmt members,
including a bold NOTE! for future developers that codes pointer
is NULL for the in-memory-only formats]

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Introduce PIXFMT_SEL_IPU

Add a PIXFMT_SEL_IPU selection flag, to select only the IPU-internal
pixel formats, and move the single-entry IPU-internal pixel format
arrays into pixel_formats[]. imx_media_find_ipu_format() and
imx_media_enum_ipu_format() can now simply call find_format() and
enum_format().

The RGB32 format is both an IPU-internal format, and an in-memory format
via idmac channels that is supported by the IPUv3 driver, so it appears
twice in pixel_formats[], one with ipufmt=false for the in-memory format,
and again with ipufmt=true for the IPU-internal format.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Rename pixel format selection enumeration

After the introduction of the CS_SEL_BAYER flag, the "codespace"
pixel format selection enumeration wording no longer makes sense
(and even before, when selecting between YUV or RGB formats,
"codespace" was a misuse of the term).

Rename

- 'enum codespace_sel' to 'enum imx_pixfmt_sel'
- CS_SEL_* to PIXFMT_SEL_*
- local vars named cs_sel to fmt_sel or just sel

No functional changes.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: Fix some pixel format selections

- imx_media_capture_device_register() needs to use CS_SEL_ANY when
finding the format from the attached source subdevice, because the
source can be a CSI which supports bayer, and the CSI may have selected
a bayer format when it registered.

- Likewise, imx_media_init_mbus_fmt() is called from the CSI, so the
function may be passed a bayer code. Use CS_SEL_ANY when locating
the format.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Handle Bayer format lookup through a selection flag

The format lookup (and enumeration) functions take a boolean flag to
tell if Bayer formats should be considered. This leads to hard to read
lines such as

return enum_format(fourcc, NULL, index, cs_sel, true, false);

where the boolean parameters can easily be mixed. To make the code
clearer, add a CS_SEL_BAYER flag that can be passed through the
codespace_sel parameter of the lookup functions to replace the bool
parameter.

[slongerbeam@gmail.com: Instead of declaring CS_SEL_ANY as a bitfield
containing only CS_SEL_YUV | CS_SEL_RGB, declare CS_SEL_ANY as all of
the above (YUV, RGB, BAYER). A new enum is declared for the YUV | RGB
selection as CS_SEL_YUV_RGB, and that is used by sub-devices that
don't support BAYER and only allow selecting and enumerating YUV or RGB
encodings. CS_SEL_ANY is now only used by the CSI sub-devices and the
attached capture interfaces, since only those devices support BAYER
formats.]

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Inline init_mbus_colorimetry() in its caller

The init_mbus_colorimetry() function is small and used in a single
place. The code becomes easier to follow if it gets inline in its
caller. Do so.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: fix media bus format enumeration

Iterate over all media bus formats, not just over the first format in
each imx_media_pixfmt entry.

Before:

$ v4l2-ctl -d $(media-ctl -e ipu1_csi0) --list-subdev-mbus-codes 0
ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=0)
0x2006: MEDIA_BUS_FMT_UYVY8_2X8
0x2008: MEDIA_BUS_FMT_YUYV8_2X8
0x1008: MEDIA_BUS_FMT_RGB565_2X8_LE
0x100a: MEDIA_BUS_FMT_RGB888_1X24
0x100d: MEDIA_BUS_FMT_ARGB8888_1X32
0x3001: MEDIA_BUS_FMT_SBGGR8_1X8
0x3013: MEDIA_BUS_FMT_SGBRG8_1X8
0x3002: MEDIA_BUS_FMT_SGRBG8_1X8
0x3014: MEDIA_BUS_FMT_SRGGB8_1X8
0x3007: MEDIA_BUS_FMT_SBGGR10_1X10
0x300e: MEDIA_BUS_FMT_SGBRG10_1X10
0x300a: MEDIA_BUS_FMT_SGRBG10_1X10
0x300f: MEDIA_BUS_FMT_SRGGB10_1X10
0x2001: MEDIA_BUS_FMT_Y8_1X8
0x200a: MEDIA_BUS_FMT_Y10_1X10

After:

$ v4l2-ctl -d $(media-ctl -e ipu1_csi0) --list-subdev-mbus-codes 0
ioctl: VIDIOC_SUBDEV_ENUM_MBUS_CODE (pad=0)
0x2006: MEDIA_BUS_FMT_UYVY8_2X8
0x200f: MEDIA_BUS_FMT_UYVY8_1X16
0x2008: MEDIA_BUS_FMT_YUYV8_2X8
0x2011: MEDIA_BUS_FMT_YUYV8_1X16
0x1008: MEDIA_BUS_FMT_RGB565_2X8_LE
0x100a: MEDIA_BUS_FMT_RGB888_1X24
0x100c: MEDIA_BUS_FMT_RGB888_2X12_LE
0x100d: MEDIA_BUS_FMT_ARGB8888_1X32
0x3001: MEDIA_BUS_FMT_SBGGR8_1X8
0x3013: MEDIA_BUS_FMT_SGBRG8_1X8
0x3002: MEDIA_BUS_FMT_SGRBG8_1X8
0x3014: MEDIA_BUS_FMT_SRGGB8_1X8
0x3007: MEDIA_BUS_FMT_SBGGR10_1X10
0x3008: MEDIA_BUS_FMT_SBGGR12_1X12
0x3019: MEDIA_BUS_FMT_SBGGR14_1X14
0x301d: MEDIA_BUS_FMT_SBGGR16_1X16
0x300e: MEDIA_BUS_FMT_SGBRG10_1X10
0x3010: MEDIA_BUS_FMT_SGBRG12_1X12
0x301a: MEDIA_BUS_FMT_SGBRG14_1X14
0x301e: MEDIA_BUS_FMT_SGBRG16_1X16
0x300a: MEDIA_BUS_FMT_SGRBG10_1X10
0x3011: MEDIA_BUS_FMT_SGRBG12_1X12
0x301b: MEDIA_BUS_FMT_SGRBG14_1X14
0x301f: MEDIA_BUS_FMT_SGRBG16_1X16
0x300f: MEDIA_BUS_FMT_SRGGB10_1X10
0x3012: MEDIA_BUS_FMT_SRGGB12_1X12
0x301c: MEDIA_BUS_FMT_SRGGB14_1X14
0x3020: MEDIA_BUS_FMT_SRGGB16_1X16
0x2001: MEDIA_BUS_FMT_Y8_1X8
0x200a: MEDIA_BUS_FMT_Y10_1X10
0x2013: MEDIA_BUS_FMT_Y12_1X12

[laurent.pinchart@ideasonboard.com: Decrement index to replace loop counter k]
[laurent.pinchart@ideasonboard.com: Return directly from within the loops]

Fixes: e130291212df5 ("[media] media: Add i.MX media core driver")
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: fix and simplify pixel format enumeration

Merge yuv_formats and rgb_formats into a single array. Always loop over
all entries, skipping those that do not match the requested search
criteria. This simplifies the code, lets us get rid of the manual
counting of array entries, and stops accidentally ignoring some non-mbus
RGB formats.

Before:

$ v4l2-ctl -d /dev/video14 --list-formats-out
ioctl: VIDIOC_ENUM_FMT
Type: Video Output

[0]: 'UYVY' (UYVY 4:2:2)
[1]: 'YUYV' (YUYV 4:2:2)
[2]: 'YU12' (Planar YUV 4:2:0)
[3]: 'YV12' (Planar YVU 4:2:0)
[4]: '422P' (Planar YUV 4:2:2)
[5]: 'NV12' (Y/CbCr 4:2:0)
[6]: 'NV16' (Y/CbCr 4:2:2)
[7]: 'RGBP' (16-bit RGB 5-6-5)
[8]: 'RGB3' (24-bit RGB 8-8-8)
[9]: 'BX24' (32-bit XRGB 8-8-8-8)

After:

$ v4l2-ctl -d /dev/video14 --list-formats-out
ioctl: VIDIOC_ENUM_FMT
Type: Video Output

[0]: 'UYVY' (UYVY 4:2:2)
[1]: 'YUYV' (YUYV 4:2:2)
[2]: 'YU12' (Planar YUV 4:2:0)
[3]: 'YV12' (Planar YVU 4:2:0)
[4]: '422P' (Planar YUV 4:2:2)
[5]: 'NV12' (Y/CbCr 4:2:0)
[6]: 'NV16' (Y/CbCr 4:2:2)
[7]: 'RGBP' (16-bit RGB 5-6-5)
[8]: 'RGB3' (24-bit RGB 8-8-8)
[9]: 'BGR3' (24-bit BGR 8-8-8)
[10]: 'BX24' (32-bit XRGB 8-8-8-8)
[11]: 'XR24' (32-bit BGRX 8-8-8-8)
[12]: 'RX24' (32-bit XBGR 8-8-8-8)
[13]: 'XB24' (32-bit RGBX 8-8-8-8)

Tested on a imx6q-sabresd.

[laurent.pinchart@ideasonboard.com: Make loop counters unsigned]
[laurent.pinchart@ideasonboard.com: Decrement index instead of adding a counter]
[laurent.pinchart@ideasonboard.com: Return directly from within loop instead of breaking]
[slongerbeam@gmail.com: Fix colorspace comparison error]

Fixes: e130291212df5 ("[media] media: Add i.MX media core driver")
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Tested-by: Fabio Estevam <festevam@gmail.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: imx6-media-csi: Support clamping Y10 and Y12 to Y8

10-bit and 12-bit greyscale input data to the CSI can be written as
8-bit data to memory. Support this.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: imx6-media-csi: Replace Y16 with Y10 and Y12

The driver doesn't really support V4L2_PIX_FMT_Y16, as there's no 16-bit
greyscale media bus code defined by the kernel. It (ab)uses the format
to capture 10-bit and 12-bit greyscale formats. Fix it to properly
support MEDIA_BUS_FMT_Y10_1X10 and MEDIA_BUS_FMT_Y12_1X12 instead.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Rui Miguel Silva <rmfrfs@gmail.com>
Acked-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: enable V4L2_PIX_FMT_XBGR32, _BGRX32, and _RGBX32

Now that proper V4L2 pixel formats exist for all 32-bit RGB
permutations, drop support for the poorly defined legacy format
V4L2_PIX_FMT_BGR32.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: Fix NULL deref in find_pipeline_entity()

Fix a cut&paste error in find_pipeline_entity(). The start entity must be
passed to media_entity_to_video_device() in find_pipeline_entity(), not
pad->entity. The pad is only put to use later, after determining the start
entity is not the entity being searched for.

Fixes: 3ef46bc97ca2 ("media: staging/imx: Improve pipeline searching")
Reported-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: imx: Try colorimetry at both sink and source pads

Retask imx_media_fill_default_mbus_fields() to try colorimetry parameters,
renaming it to to imx_media_try_colorimetry(), and call it at both sink and
source pad try_fmt's. The unrelated check for uninitialized field value is
moved out to appropriate places in each subdev try_fmt.

The IC now supports Rec.709 and BT.601 Y'CbCr encoding, and both limited
and full range quantization for both YUV and RGB space, so allow those
for pipelines that route through the IC.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Acked-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>

media: staging/imx: Improve pipeline searching

Export find_pipeline_pad(), renaming to imx_media_pipeline_pad(), and
extend its functionality to allow searching for video devices in the
enabled pipeline in addition to sub-devices.

As part of this:

- Rename imx_media_find_mipi_csi2_channel() to
imx_media_pipeline_csi2_channel().

- Remove imx_media_find_upstream_pad(), it is redundant now.

- Rename imx_media_find_upstream_subdev() to imx_media_pipeline_subdev()
with an additional boolean argument for searching upstream or downstream.

- Add imx_media_pipeline_video_device() which is analogous to
imx_media_pipeline_subdev() but searches for video devices.

- Remove imxmd pointer arg from all of the functions above, it was
never used in those functions. With that change the i.MX5/6 CSI,
VDIC, and IC sub-devices no longer require the media_device.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: Revert "media: imx: Set capture compose rectangle in capture_device_set_format"

Revert this commit, as imx_media_capture_device_set_format() will be
removed. The arguments to mx_media_mbus_fmt_to_pix_fmt() and
imx_media_capture_device_set_format() in imx7_csi_set_fmt() are also
reverted.

This reverts commit 5964cbd8692252615370b77eb96764dd70c2f837.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: Move add_video_device into capture_device_register

Move imx_media_add_video_device() into imx_media_capture_device_register().
Also the former has no error conditions to convert to void.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: Pass device to alloc/free_dma_buf

Allocate and free a DMA coherent buffer in imx_media_alloc/free_dma_buf()
from the given device. This allows DMA alloc and free using a device
that is backed by real hardware, which for the imx5/6/7 CSI is the CSI
unit, and for the internal IPU sub-devices, is the parent IPU.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

staging: media: imx: add proper SPDX identifiers on files that did not have them.

There were a few files for the imx media drivers that did not have SPDX
identifiers on them, so fix that up. At the same time, remove the "free
form" text that specified the license of the file, as that is impossible
for any tool to properly parse.

Cc: Philipp Zabel <p.zabel@pengutronix.de>
Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Shawn Guo <shawnguo@kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Fabio Estevam <festevam@gmail.com>
Cc: NXP Linux Team <linux-imx@nxp.com>
Cc: linux-media@vger.kernel.org
Acked-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

media: staging/imx: rearrange group id to take in account IPU

Some imx system do not have IPU, so prepare the imx media drivers to
support this kind of devices. Rename the group ids to include an _IPU_
prefix, add a new group id to support systems with only a CSI without
IPU, and also rename the create internal links to make it clear that
only systems with IPU have internal subdevices.

Signed-off-by: Rui Miguel Silva <rui.silva@linaro.org>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: imx: Set capture compose rectangle in capture_device_set_format

The capture compose rectangle was not getting updated when setting
the source subdevice's source pad format. This causes the compose window
to be zero (or not updated) at stream start unless the capture device
format was set explicitly at the capture device node.

Fix by moving the calculation of the capture compose rectangle to
imx_media_mbus_fmt_to_pix_fmt(), and pass the rectangle to
imx_media_capture_device_set_format().

Fixes: 439d8186fb23 ("media: imx: add capture compose rectangle")

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: imx: lift CSI and PRP ENC/VF width alignment restriction

The CSI, PRP ENC, and PRP VF subdevices shouldn't have to care about
IDMAC line start address alignment. With compose rectangle support in
the capture driver, they don't have to anymore.
If the direct CSI -> IC path is enabled, the CSI output width must
still be aligned to 8 pixels (IC burst length).

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: imx: use well defined 32-bit RGB pixel format

The documentation in Documentation/media/uapi/v4l/pixfmt-packed-rgb.rst
tells us that the V4L2_PIX_FMT_RGB32 format is deprecated and must not
be used by new drivers. Replace it with V4L2_PIX_FMT_XRGB32.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Acked-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: imx: add support for RGB565_2X8 on parallel bus

The IPU can only capture RGB565 with two 8-bit cycles in bayer/generic
mode on the parallel bus, compared to a specific mode on MIPI CSI-2.
To handle this, we extend imx_media_pixfmt with a cycles per pixel
field, which is used for generic formats on the parallel bus.

Based on the selected format and bus, we then update the width to
account for the multiple cycles per pixel.

Signed-off-by: Jan Luebbe <jlu@pengutronix.de>
Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: imx: add 16-bit grayscale support

Since commit 50b0f0aee839 ("gpu: ipu-csi: add 10/12-bit grayscale
support to mbus_code_to_bus_cfg") the IPU CSI can be configured to
capture 10-bit and 12-bit grayscale formats, expanded to 16-bit
grayscale, in bayer/generic data mode.
This patch adds support for V4L2_PIX_FMT_Y16 captured from sensors
that provide MEDIA_BUS_FMT_Y10_1X10 or MEDIA_BUS_FMT_Y12_1X12 data.

Cc: Jan Luebbe <jlu@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Acked-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: imx-media-utils: fix a warning

The logic at find_format() is a little bit confusing even for
humans, and it tricks static code analyzers:

drivers/staging/media/imx/imx-media-utils.c:259 find_format() error: buffer overflow 'array' 14 <= 20

Rewrite the logic in a way that it makes it clearer to understand,
while prevent static analyzers to produce false positives.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: imx: add 8-bit grayscale support

The IPUv3 code has 8-bit grayscale capture support.
Enable imx-media to use it.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging/imx: Implement init_cfg subdev pad op

Implement the init_cfg pad op in all imx-media subdevices. The try
formats are initialized to the current active formats on all pads.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging/imx: convert static vdev lists to list_head

Although not technically necessary because imx-media has only a
maximum of 8 video devices, and once setup the video device lists
are static, in anticipation of moving control ineritance to
v4l2-core, make the vdev lists more generic by converting to
dynamic list_head's.

After doing that, 'struct imx_media_pad' is now just a list_head
of video devices reachable from a pad. Allocate an array of list_head's,
one list_head for each pad, and attach that array to sd->host_priv.
An entry in the pad lists is of type 'struct imx_media_pad_vdev', and
points to a video device from the master list.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging/imx: remove static subdev arrays

For more complex OF graphs, there will be more async subdevices
registered. Remove the static subdev[IMX_MEDIA_MAX_SUBDEVS] array,
so that imx-media places no limits on the number of async subdevs
that can be added and registered.

There were two uses for 'struct imx_media_subdev'. First was to act
as the async subdev list to be passed to v4l2_async_notifier_register().

Second was to aid in inheriting subdev controls to the capture devices,
and this is done by creating a list of capture devices that can be reached
from a subdev's source pad. So 'struct imx_media_subdev' also contained
a static array of 'struct imx_media_pad' for placing the capture device
lists at each pad.

'struct imx_media_subdev' has been completely removed. Instead, at async
completion, allocate an array of 'struct imx_media_pad' and attach it to
the subdev's host_priv pointer, in order to support subdev controls
inheritance.

Likewise, remove static async_ptrs[IMX_MEDIA_MAX_SUBDEVS] array.
Instead, allocate a 'struct imx_media_async_subdev' when forming
the async list, and add it to an asd_list list_head in
imx_media_add_async_subdev(). At async completion, allocate the
asd pointer list and pull the asd's off asd_list for
v4l2_async_notifier_register().

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging/imx: get CSI bus type from nearest upstream entity

The imx-media driver currently supports a device tree graph of
limited complexity. This patch is a first step in allowing imx-media
to work with more general OF graphs.

The CSI subdevice assumes the originating upstream subdevice (the
"sensor") is connected directly to either the CSI mux or the MIPI
CSI-2 receiver. But for more complex graphs, the sensor can be distant,
with possible bridge entities in between. Thus the sensor's bus type
could be quite different from what is entering the CSI. For example
a distant sensor could have a parallel interface, but the stream
entering the i.MX is MIPI CSI-2.

To remove this assumption, get the entering bus config from the entity
that is directly upstream from either the CSI mux, or the CSI-2 receiver.
If the CSI-2 receiver is not in the enabled pipeline, the bus type to the
CSI is parallel, otherwise the CSI is receiving MIPI CSI-2.

Note that we can't use the direct upstream source connected to CSI
(which is either the CSI mux or the CSI-2 receiver) to determine
bus type. The bus entering the CSI from the CSI-2 receiver is a 32-bit
parallel bus containing the demultiplexed MIPI CSI-2 virtual channels.
But the CSI and its IDMAC channels must be configured based on whether
it is receiving data from the CSI-2 receiver or from the CSI mux's
parallel interface pins.

The function csi_get_upstream_endpoint() is used to find this
endpoint. It makes use of a new utility function
imx_media_find_upstream_pad(), that if given a grp_id of 0, will
return the closest upstream pad from start_entity.

With these changes, imx_media_find_sensor() is no longer used and
is removed. As a result there is also no longer a need to identify
any sensor or set the sensor subdev's group id as a method to search
for it. So IMX_MEDIA_GRP_ID_SENSOR is removed. Also the video-mux group
id IMX_MEDIA_GRP_ID_VIDMUX was never used so that is removed as well.
The remaining IMX_MEDIA_GRP_ID_* definitions are entities internal
to the i.MX.

Another use of imx_media_find_sensor() in the CSI was to call the
sensor's g_skip_frames op to determine if a delay was needed before
enabling the CSI at stream on. If necessary this will have to be
re-addressed at a later time.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

[media] media: imx: set and propagate default field, colorimetry

This patch adds a call to imx_media_fill_default_mbus_fields() in the
*_try_fmt() functions at the sink pads, to set empty field order and
colorimetry parameters.

If the field order is set to ANY, choose the currently set field order
at the sink pad. If the colorspace is set to DEFAULT, choose the
current colorspace at the sink pad. If any of xfer_func, ycbcr_enc
or quantization are set to DEFAULT, either choose the current sink pad
setting, or the default setting for the new colorspace, if non-DEFAULT
colorspace was given.

If a format is destined to be routed through the Image Converter,
fixed quantization and Y`CbCr encoding must be set.

Colorimetry is also propagated from sink to source pads anywhere
this has not already been done.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

[media] media: Add i.MX media core driver

Add the core media driver for i.MX SOC.

Switch from the v4l2_of_ APIs to the v4l2_fwnode_ APIs.
Add the bayer formats to imx-media's list of supported pixel and bus
formats.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>