media: v4l2-subdev: Rename .init_cfg() operation to .init_state()

The subdev .init_cfg() operation is affected by two issues:

- It has long been extended to initialize a whole v4l2_subdev_state
instead of just a v4l2_subdev_pad_config, but its name has stuck
around.

- Despite operating on a whole subdev state and not being directly
exposed to the subdev users (either in-kernel or through the userspace
API), .init_cfg() is categorized as a subdev pad operation.

This participates in making the subdev API confusing for new developers.
Fix it by renaming the operation to .init_state(), and make it a subdev
internal operation.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Michael Riesch <michael.riesch@wolfvision.net> # for imx415
Acked-by: Shuah Khan <skhan@linuxfoundation.org> # for vimc
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
[Sakari Ailus: Resolved a conflict in Renesas vsp1 driver.]
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

staging: media: imx: Make imx_media_of_add_csi() static

The imx_media_of_add_csi() function is only called in its compilation
unit. Make it static and don't export it.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

staging: media: imx: Drop unused helper functions

The imx_media_find_subdev_by_fwnode(),
imx_media_find_subdev_by_devname(), imx_media_pipeline_csi2_channel()
and imx_media_pipeline_video_device() helper functions are not used.
Drop them.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

staging: media: imx: Drop IMX_MEDIA_GRP_ID_CSI

The IMX_MEDIA_GRP_ID_CSI group ID isn't used anymore. Drop it.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

media: staging: media: imx: change imx_media_fim_set_stream() to return void

At present, the function imx_media_fim_set_stream() always returns 0.
So, convert it to be a function returning void instead. Issue identified
using the returnvar.cocci Coccinelle semantic patch.

Signed-off-by: Deepak R Varma <drv@mailo.com>
Reviewed-by: Marco Felsch <m.felsch@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

media: imx: Use get_mbus_config instead of parsing upstream DT endpoints

Stop parsing upstream neighbors' device-tree endpoints to retrieve the
media bus configuration. Instead use the get_mbus_config op and throw an
error if the upstream subdevice does not implement it.

Also drop the corresponding TODO entry and the now unused
imx_media_get_pad_fwnode() function.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Tested-by: Marco Felsch <m.felsch@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>

media: imx: Remove unused functions

Neither imx_media_mbus_fmt_to_ipu_image nor imx_media_ipu_image_to_mbus_fmt
were used anywhere.

Signed-off-by: Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: v4l2-subdev: add subdev-wide state struct

We have 'struct v4l2_subdev_pad_config' which contains configuration for
a single pad used for the TRY functionality, and an array of those
structs is passed to various v4l2_subdev_pad_ops.

I was working on subdev internal routing between pads, and realized that
there's no way to add TRY functionality for routes, which is not pad
specific configuration. Adding a separate struct for try-route config
wouldn't work either, as e.g. set-fmt needs to know the try-route
configuration to propagate the settings.

This patch adds a new struct, 'struct v4l2_subdev_state' (which at the
moment only contains the v4l2_subdev_pad_config array) and the new
struct is used in most of the places where v4l2_subdev_pad_config was
used. All v4l2_subdev_pad_ops functions taking v4l2_subdev_pad_config
are changed to instead take v4l2_subdev_state.

The changes to drivers/media/v4l2-core/v4l2-subdev.c and
include/media/v4l2-subdev.h were written by hand, and all the driver
changes were done with the semantic patch below. The spatch needs to be
applied to a select list of directories. I used the following shell
commands to apply the spatch:

dirs="drivers/media/i2c drivers/media/platform drivers/media/usb drivers/media/test-drivers/vimc drivers/media/pci drivers/staging/media"
for dir in $dirs; do spatch -j8 --dir --include-headers --no-show-diff --in-place --sp-file v4l2-subdev-state.cocci $dir; done

Note that Coccinelle chokes on a few drivers (gcc extensions?). With
minor changes we can make Coccinelle run fine, and these changes can be
reverted after spatch. The diff for these changes is:

For drivers/media/i2c/s5k5baf.c:

@@ -1481,7 +1481,7 @@ static int s5k5baf_set_selection(struct v4l2_subdev *sd,
&s5k5baf_cis_rect,
v4l2_subdev_get_try_crop(sd, cfg, PAD_CIS),
v4l2_subdev_get_try_compose(sd, cfg, PAD_CIS),
- v4l2_subdev_get_try_crop(sd, cfg, PAD_OUT)
+ v4l2_subdev_get_try_crop(sd, cfg, PAD_OUT),
};
s5k5baf_set_rect_and_adjust(rects, rtype, &sel->r);
return 0;

For drivers/media/platform/s3c-camif/camif-capture.c:

@@ -1230,7 +1230,7 @@ static int s3c_camif_subdev_get_fmt(struct v4l2_subdev *sd,
*mf = camif->mbus_fmt;
break;

- case CAMIF_SD_PAD_SOURCE_C...CAMIF_SD_PAD_SOURCE_P:
+ case CAMIF_SD_PAD_SOURCE_C:
/* crop rectangle at camera interface input */
mf->width = camif->camif_crop.width;
mf->height = camif->camif_crop.height;
@@ -1332,7 +1332,7 @@ static int s3c_camif_subdev_set_fmt(struct v4l2_subdev *sd,
}
break;

- case CAMIF_SD_PAD_SOURCE_C...CAMIF_SD_PAD_SOURCE_P:
+ case CAMIF_SD_PAD_SOURCE_C:
/* Pixel format can be only changed on the sink pad. */
mf->code = camif->mbus_fmt.code;
mf->width = crop->width;

The semantic patch is:

// <smpl>

// Change function parameter

@@
identifier func;
identifier cfg;
@@

func(...,
- struct v4l2_subdev_pad_config *cfg
+ struct v4l2_subdev_state *sd_state
, ...)
{
<...
- cfg
+ sd_state
...>
}

// Change function declaration parameter

@@
identifier func;
identifier cfg;
type T;
@@
T func(...,
- struct v4l2_subdev_pad_config *cfg
+ struct v4l2_subdev_state *sd_state
, ...);

// Change function return value

@@
identifier func;
@@
- struct v4l2_subdev_pad_config
+ struct v4l2_subdev_state
*func(...)
{
...
}

// Change function declaration return value

@@
identifier func;
@@
- struct v4l2_subdev_pad_config
+ struct v4l2_subdev_state
*func(...);

// Some drivers pass a local pad_cfg for a single pad to a called function. Wrap it
// inside a pad_state.

@@
identifier func;
identifier pad_cfg;
@@
func(...)
{
...
struct v4l2_subdev_pad_config pad_cfg;
+ struct v4l2_subdev_state pad_state = { .pads = &pad_cfg };

<+...

(
v4l2_subdev_call
|
sensor_call
|
isi_try_fse
|
isc_try_fse
|
saa_call_all
)
(...,
- &pad_cfg
+ &pad_state
,...)

...+>
}

// If the function uses fields from pad_config, access via state->pads

@@
identifier func;
identifier state;
@@
func(...,
struct v4l2_subdev_state *state
, ...)
{
<...
(
- state->try_fmt
+ state->pads->try_fmt
|
- state->try_crop
+ state->pads->try_crop
|
- state->try_compose
+ state->pads->try_compose
)
...>
}

// If the function accesses the filehandle, use fh->state instead

@@
struct v4l2_subdev_fh *fh;
@@
- fh->pad
+ fh->state

@@
struct v4l2_subdev_fh fh;
@@
- fh.pad
+ fh.state

// Start of vsp1 specific

@@
@@
struct vsp1_entity {
...
- struct v4l2_subdev_pad_config *config;
+ struct v4l2_subdev_state *config;
...
};

@@
symbol entity;
@@
vsp1_entity_init(...)
{
...
entity->config =
- v4l2_subdev_alloc_pad_config
+ v4l2_subdev_alloc_state
(&entity->subdev);
...
}

@@
symbol entity;
@@
vsp1_entity_destroy(...)
{
...
- v4l2_subdev_free_pad_config
+ v4l2_subdev_free_state
(entity->config);
...
}

@exists@
identifier func =~ "(^vsp1.*)|(hsit_set_format)|(sru_enum_frame_size)|(sru_set_format)|(uif_get_selection)|(uif_set_selection)|(uds_enum_frame_size)|(uds_set_format)|(brx_set_format)|(brx_get_selection)|(histo_get_selection)|(histo_set_selection)|(brx_set_selection)";
symbol config;
@@
func(...) {
...
- struct v4l2_subdev_pad_config *config;
+ struct v4l2_subdev_state *config;
...
}

// End of vsp1 specific

// Start of rcar specific

@@
identifier sd;
identifier pad_cfg;
@@
rvin_try_format(...)
{
...
- struct v4l2_subdev_pad_config *pad_cfg;
+ struct v4l2_subdev_state *sd_state;
...
- pad_cfg = v4l2_subdev_alloc_pad_config(sd);
+ sd_state = v4l2_subdev_alloc_state(sd);
<...
- pad_cfg
+ sd_state
...>
- v4l2_subdev_free_pad_config(pad_cfg);
+ v4l2_subdev_free_state(sd_state);
...
}

// End of rcar specific

// Start of rockchip specific

@@
identifier func =~ "(rkisp1_rsz_get_pad_fmt)|(rkisp1_rsz_get_pad_crop)|(rkisp1_rsz_register)";
symbol rsz;
symbol pad_cfg;
@@

func(...)
{
+ struct v4l2_subdev_state state = { .pads = rsz->pad_cfg };
...
- rsz->pad_cfg
+ &state
...
}

@@
identifier func =~ "(rkisp1_isp_get_pad_fmt)|(rkisp1_isp_get_pad_crop)";
symbol isp;
symbol pad_cfg;
@@

func(...)
{
+ struct v4l2_subdev_state state = { .pads = isp->pad_cfg };
...
- isp->pad_cfg
+ &state
...
}

@@
symbol rkisp1;
symbol isp;
symbol pad_cfg;
@@

rkisp1_isp_register(...)
{
+ struct v4l2_subdev_state state = { .pads = rkisp1->isp.pad_cfg };
...
- rkisp1->isp.pad_cfg
+ &state
...
}

// End of rockchip specific

// Start of tegra-video specific

@@
identifier sd;
identifier pad_cfg;
@@
__tegra_channel_try_format(...)
{
...
- struct v4l2_subdev_pad_config *pad_cfg;
+ struct v4l2_subdev_state *sd_state;
...
- pad_cfg = v4l2_subdev_alloc_pad_config(sd);
+ sd_state = v4l2_subdev_alloc_state(sd);
<...
- pad_cfg
+ sd_state
...>
- v4l2_subdev_free_pad_config(pad_cfg);
+ v4l2_subdev_free_state(sd_state);
...
}

@@
identifier sd_state;
@@
__tegra_channel_try_format(...)
{
...
struct v4l2_subdev_state *sd_state;
<...
- sd_state->try_crop
+ sd_state->pads->try_crop
...>
}

// End of tegra-video specific

// </smpl>

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: capture: Support creating immutable link to capture device

When the subdevice connected to the capture device has a single possible
sink, there's no point in making the link mutable. Support creating
immutable links.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: capture: Add a mechanism to disable control inheritance

Add a parameter to the imx_media_capture_device_init() function to
select between the legacy and MC-centric API. When selecting the
MC-centric API, the video node doesn't inherit controls from subdevs
anymore. All callers are updated to use the legacy API for now,
preserving the existing behaviour.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: capture: Store v4l2_pix_format in imx_media_video_dev

The imx_media_video_dev structure stores the active format on the video
node in an instance of v4l2_format. This wastes memory when all we need
is the information contained in the smaller v4l2_pix_format
sub-structure. Replace v4l2_format by v4l2_pix_format to save memory.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Rui Miguel Silva <rmfrfs@gmail.com>
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Add ability to filter pixel formats by mbus code

Add a media bus code argument to the imx_media_enum_pixel_formats(). If
set to a non-zero value, the function will only consider pixel formats
that match the given media bus code.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: Set default sizes through macros in all drivers

All drivers use 640x480 as the default size, but they all hardcode those
values. Create two global macros named IMX_MEDIA_DEF_PIX_WIDTH and
IMX_MEDIA_DEF_PIX_HEIGHT to store the default size, and use them through
the code.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: staging/imx: Increase IMX_MEDIA_EOF_TIMEOUT

When trying to capture video on a imx6dl-based board with an ADV7280,
the following timeout error is observed:

v4l2-ctl --stream-mmap -d /dev/video2
[ 22.792049] ipu1_csi1: EOF timeout
VIDIOC_DQBUF: failed: Input/output error

Increase the IMX_MEDIA_EOF_TIMEOUT to avoid such problem.

Signed-off-by: Fabio Estevam <festevam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: Create missing links from CSI-2 receiver

The entities external to the i.MX6 IPU and i.MX7 now create the links
to their fwnode-endpoint connected entities in their notifier bound
callbacks. Which means imx_media_create_of_links() and
imx_media_create_csi_of_links() are no longer needed and are removed.

However there is still one case in which imx-media needs to create
fwnode-endpoint based links at probe completion. The v4l2-async framework
does not allow multiple subdevice notifiers to contain a duplicate
subdevice in their asd_list. Only the first subdev notifier that discovers
and adds that one subdevice to its asd_list will receive a bound callback
for it. Other subdevices that also have firmware endpoint connections to
this duplicate subdevice will not have it in their asd_list, and thus will
never receive a bound callback for it. In the case of imx-media, the one
duplicate subdevice in question is the i.MX6 MIPI CSI-2 receiver.

Until there is a solution to that problem, rewrite imx_media_create_links()
to add the missing links from the CSI-2 receiver to the CSIs and CSI muxes.
The function is renamed imx_media_create_csi2_links().

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: csi: Lookup upstream endpoint with imx_media_get_pad_fwnode

Fix the 1:1 port-id:pad-index assumption for the upstream subdevice, by
searching the upstream subdevice's endpoints for one that maps to the
pad's index. This is carried out by a new reverse mapping function
imx_media_get_pad_fwnode().

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx5/6/7: csi: Mark a bound video mux as a CSI mux

For i.MX5/6, if the bound subdev is a video mux, it must be one of the
CSI muxes, and for i.MX7, the bound subdev must always be a CSI mux.

So if the bound subdev is a video mux, mark it as a CSI mux with a new
group id IMX_MEDIA_GRP_ID_CSI_MUX.

In the process use the new group id in csi_get_upstream_endpoint(), and
do some cleanup in that function for better readability.

Suggested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Constify some mbus and ipu_image arguments

The imx_media_mbus_fmt_to_pix_fmt() and imx_media_mbus_fmt_to_ipu_image()
functions do not need to modify their mbus argument, and
imx_media_ipu_image_to_mbus_fmt() does not need to modify its ipu_image
argument. Make them const.

[slongerbeam@gmail.com: Constified mbus arg to
imx_media_mbus_fmt_to_ipu_image(), and ipu_image
arg to imx_media_ipu_image_to_mbus_fmt(), as well]

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Rename format lookup and enumeration functions

Rename the format lookup and enumeration functions according to their
usage:

- Rename imx_media_(find|enum)_format() to *_pixel_format() to
explicitly state on what formats the functions operate. This aligns
the naming scheme with the media bus and IPU format functions that
already end with *_mbus_format() and *_ipu_formats().

- Rename all enumeration functions to pluralize 'formats' at the end, as
they enumerate multiple formats.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Split find|enum_format into fourcc and mbus functions

To make the code easier to follow, split up find_format() into separate
search functions for pixel formats and media-bus codes, and inline
find_format() into the exported functions imx_media_find_format()
and imx_media_find_mbus_format().

Do the equivalent for enum_format().

Also add comment blocks for the exported find|enum functions.

The convenience functions imx_media_find_ipu_format() and
imx_media_enum_ipu_format() can now be made inline and moved to
imx-media.h.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Make imx_media_pixfmt handle variable number of codes

The imx_media_pixfmt structures include a codes member that stores
media bus codes as a fixed array of 4 integers. The functions dealing
with the imx_media_pixfmt structures assume that the array of codes is
terminated by a 0 element. This mechanism is fragile, as demonstrated
by several instances of the structure containing 4 non-zero codes.

Fix this by turning the array into a pointer, and providing an
IMX_BUS_FMTS macro to initialize the codes member with a guaranteed 0
element at the end.

[Fixed a NULL deref of the codes pointer in a couple places]
[Added more comments for the struct imx_media_pixfmt members,
including a bold NOTE! for future developers that codes pointer
is NULL for the in-memory-only formats]

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Introduce PIXFMT_SEL_IPU

Add a PIXFMT_SEL_IPU selection flag, to select only the IPU-internal
pixel formats, and move the single-entry IPU-internal pixel format
arrays into pixel_formats[]. imx_media_find_ipu_format() and
imx_media_enum_ipu_format() can now simply call find_format() and
enum_format().

The RGB32 format is both an IPU-internal format, and an in-memory format
via idmac channels that is supported by the IPUv3 driver, so it appears
twice in pixel_formats[], one with ipufmt=false for the in-memory format,
and again with ipufmt=true for the IPU-internal format.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Rename pixel format selection enumeration

After the introduction of the CS_SEL_BAYER flag, the "codespace"
pixel format selection enumeration wording no longer makes sense
(and even before, when selecting between YUV or RGB formats,
"codespace" was a misuse of the term).

Rename

- 'enum codespace_sel' to 'enum imx_pixfmt_sel'
- CS_SEL_* to PIXFMT_SEL_*
- local vars named cs_sel to fmt_sel or just sel

No functional changes.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: utils: Handle Bayer format lookup through a selection flag

The format lookup (and enumeration) functions take a boolean flag to
tell if Bayer formats should be considered. This leads to hard to read
lines such as

return enum_format(fourcc, NULL, index, cs_sel, true, false);

where the boolean parameters can easily be mixed. To make the code
clearer, add a CS_SEL_BAYER flag that can be passed through the
codespace_sel parameter of the lookup functions to replace the bool
parameter.

[slongerbeam@gmail.com: Instead of declaring CS_SEL_ANY as a bitfield
containing only CS_SEL_YUV | CS_SEL_RGB, declare CS_SEL_ANY as all of
the above (YUV, RGB, BAYER). A new enum is declared for the YUV | RGB
selection as CS_SEL_YUV_RGB, and that is used by sub-devices that
don't support BAYER and only allow selecting and enumerating YUV or RGB
encodings. CS_SEL_ANY is now only used by the CSI sub-devices and the
attached capture interfaces, since only those devices support BAYER
formats.]

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

media: imx: add csc/scaler mem2mem device

Add a single imx-media mem2mem video device that uses the IPU IC PP
(image converter post processing) task for scaling and colorspace
conversion.
On i.MX6Q/DL SoCs with two IPUs currently only the first IPU is used.

The hardware only supports writing to destination buffers up to
1024x1024 pixels in a single pass, arbitrary sizes can be achieved
by rendering multiple tiles per frame.

[slongerbeam@gmail.com: use ipu_image_convert_adjust(), fix
device_run() error handling, add missing media-device header,
unregister and remove the mem2mem device in error paths in
imx_media_probe_complete() and in imx_media_remove(), updated
for sync subdev registration]

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
[hverkuil-cisco@xs4all.nl: correct two minor checkpatch issues]
[hverkuil-cisco@xs4all.nl: sparse warning: make imx6_media_probe_complete static]
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: imx: Try colorimetry at both sink and source pads

Retask imx_media_fill_default_mbus_fields() to try colorimetry parameters,
renaming it to to imx_media_try_colorimetry(), and call it at both sink and
source pad try_fmt's. The unrelated check for uninitialized field value is
moved out to appropriate places in each subdev try_fmt.

The IC now supports Rec.709 and BT.601 Y'CbCr encoding, and both limited
and full range quantization for both YUV and RGB space, so allow those
for pipelines that route through the IC.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Acked-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>

media: staging/imx: Improve pipeline searching

Export find_pipeline_pad(), renaming to imx_media_pipeline_pad(), and
extend its functionality to allow searching for video devices in the
enabled pipeline in addition to sub-devices.

As part of this:

- Rename imx_media_find_mipi_csi2_channel() to
imx_media_pipeline_csi2_channel().

- Remove imx_media_find_upstream_pad(), it is redundant now.

- Rename imx_media_find_upstream_subdev() to imx_media_pipeline_subdev()
with an additional boolean argument for searching upstream or downstream.

- Add imx_media_pipeline_video_device() which is analogous to
imx_media_pipeline_subdev() but searches for video devices.

- Remove imxmd pointer arg from all of the functions above, it was
never used in those functions. With that change the i.MX5/6 CSI,
VDIC, and IC sub-devices no longer require the media_device.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: Re-organize modules

Re-organize modules, and which objects are linked into those modules, so
that:

- imx6-media (renamed from imx-media) is the media driver module for
imx5/6 only, and has no symbol exports.

- imx6-media-csi (renamed from imx-media-csi) is the subdev driver
module for imx5/6 CSI. It is now linked direcly with imx-media-fim,
since only the imx5/6 CSI makes use of the frame interval monitor.

- imx-media-common now only contains common code between imx5/6 and imx7
media drivers. It contains imx-media-utils, imx-media-of,
imx-media-dev-common, and imx-media-capture. In order to acheive that,
some functions common to imx5/6 and imx7 have been moved out of
imx-media-dev.c and into imx-media-dev-common.c.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: Remove capture_device_set_format

Don't propagate the source pad format to the connected capture device.
It's now the responsibility of userspace to call VIDIOC_S_FMT on the
capture device to ensure the capture format and compose rectangle
are compatible with the connected source. To check this, validate
the capture format with the source before streaming starts.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: Revert "media: imx: Set capture compose rectangle in capture_device_set_format"

Revert this commit, as imx_media_capture_device_set_format() will be
removed. The arguments to mx_media_mbus_fmt_to_pix_fmt() and
imx_media_capture_device_set_format() in imx7_csi_set_fmt() are also
reverted.

This reverts commit 5964cbd8692252615370b77eb96764dd70c2f837.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: Move add_video_device into capture_device_register

Move imx_media_add_video_device() into imx_media_capture_device_register().
Also the former has no error conditions to convert to void.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: Pass device to alloc/free_dma_buf

Allocate and free a DMA coherent buffer in imx_media_alloc/free_dma_buf()
from the given device. This allows DMA alloc and free using a device
that is backed by real hardware, which for the imx5/6/7 CSI is the CSI
unit, and for the internal IPU sub-devices, is the parent IPU.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: Switch to sync registration for IPU subdevs

Because the IPU sub-devices VDIC and IC are not present in the
device-tree, platform devices were created for them instead. This
allowed these sub-devices to be added to the media device's async
notifier and registered asynchronously along with the other
sub-devices that do have a device-tree presence (CSI and devices
external to the IPU and SoC).

But that approach isn't really necessary. The IPU sub-devices don't
actually require a backing device (sd->dev is allowed to be NULL).
And that approach can't get around the fact that the IPU sub-devices
are not part of a device hierarchy, which makes it awkward to retrieve
the parent IPU of these devices.

By registering them synchronously, they can be registered from the CSI
async bound notifier, so the init function for them can be given the CSI
subdev, who's dev->parent is the IPU. That is a somewhat cleaner way
to retrieve the parent IPU.

So convert to synchronous registration for the VDIC and IC task
sub-devices, at the time a CSI sub-device is bound. There is no longer
a backing device for them (sd->dev is NULL), but that's ok. Also
set the VDIC/IC sub-device owner as the IPU, so that a reference can
be taken on the IPU module.

Since the VDIC and IC task drivers are no longer platform drivers,
they are now statically linked to imx-media module.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: Revert "media: staging/imx: add media device to capture register"

The imx6-specific subdevs that register a capture device will no
longer hold a reference to the media device, so this commit must be
reverted.

This reverts commit 16204b8a1c1af77725533b77936e6c73953486ae.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: add media device to capture register

When register the capture media device it is assumed that the device
data is the media device. In the imx6 case is but in the imx7 is not
case. The device data is the csi structure.

Add the explicit argument of the media device that we want to
associate with the capture device.

Reported-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Rui Miguel Silva <rui.silva@linaro.org>
Acked-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
[hverkuil-cisco@xs4all.nl: fix checkpatch alignment warning]
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

staging: media: imx: add proper SPDX identifiers on files that did not have them.

There were a few files for the imx media drivers that did not have SPDX
identifiers on them, so fix that up. At the same time, remove the "free
form" text that specified the license of the file, as that is impossible
for any tool to properly parse.

Cc: Philipp Zabel <p.zabel@pengutronix.de>
Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Shawn Guo <shawnguo@kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Fabio Estevam <festevam@gmail.com>
Cc: NXP Linux Team <linux-imx@nxp.com>
Cc: linux-media@vger.kernel.org
Acked-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

media: imx: Don't register IPU subdevs/links if CSI port missing

The second IPU internal sub-devices were being registered and links
to them created even when the second IPU is not present. This is wrong
for i.MX6 S/DL and i.MX53 which have only a single IPU.

Fixes: e130291212df5 ("[media] media: Add i.MX media core driver")

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Cc: stable@vger.kernel.org
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: imx: Rename functions that add IPU-internal subdevs

For the functions that add and remove the internal IPU subdevice
descriptors, rename them to make clear they are the subdevs internal
to the IPU. Also rename the platform data structure for the internal
IPU subdevices. No functional changes.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Acked-by: Philipp Zabel <p.zabel@pengutronix.de>
Cc: stable@vger.kernel.org
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: rearrange group id to take in account IPU

Some imx system do not have IPU, so prepare the imx media drivers to
support this kind of devices. Rename the group ids to include an _IPU_
prefix, add a new group id to support systems with only a CSI without
IPU, and also rename the create internal links to make it clear that
only systems with IPU have internal subdevices.

Signed-off-by: Rui Miguel Silva <rui.silva@linaro.org>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: refactor imx media device probe

Refactor and move media device initialization code to a new common
module, so it can be used by other devices, this will allow for example
a near to introduce imx7 CSI driver, to use this media device.

Signed-off-by: Rui Miguel Silva <rui.silva@linaro.org>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: imx: Set capture compose rectangle in capture_device_set_format

The capture compose rectangle was not getting updated when setting
the source subdevice's source pad format. This causes the compose window
to be zero (or not updated) at stream start unless the capture device
format was set explicitly at the capture device node.

Fix by moving the calculation of the capture compose rectangle to
imx_media_mbus_fmt_to_pix_fmt(), and pass the rectangle to
imx_media_capture_device_set_format().

Fixes: 439d8186fb23 ("media: imx: add capture compose rectangle")

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: imx: add capture compose rectangle

Allowing to compose captured images into larger memory buffers
will let us lift alignment restrictions on CSI crop width.

For now all compose rectangles are identical to the complete
frame width / height. This will be changed in the next patches.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: Switch to v4l2_async_notifier_add_*_subdev

Switch to v4l2_async_notifier_add_*_subdev() when adding async subdevs
to the imx-media root notifier. This removes the need to check for
an already added asd, since v4l2_async_notifier_add_*_subdev() does this
check. Also no need to allocate a subdevs array when registering the
root notifier, or keeping an internal master asd_list, since this is
moved to the notifier's asd_list.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: Rename root notifier

Rename the imx-media root async notifier from "subdev_notifier" to
simply "notifier", so as not to confuse it with true subdev notifiers.
No functional changes.

Signed-off-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: imx: add support for RGB565_2X8 on parallel bus

The IPU can only capture RGB565 with two 8-bit cycles in bayer/generic
mode on the parallel bus, compared to a specific mode on MIPI CSI-2.
To handle this, we extend imx_media_pixfmt with a cycles per pixel
field, which is used for generic formats on the parallel bus.

Based on the selected format and bus, we then update the width to
account for the multiple cycles per pixel.

Signed-off-by: Jan Luebbe <jlu@pengutronix.de>
Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

media: staging/imx: Implement init_cfg subdev pad op

Implement the init_cfg pad op in all imx-media subdevices. The try
formats are initialized to the current active formats on all pads.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging/imx: reorder function prototypes

Re-order some of the function prototypes in imx-media.h to
group them correctly by source file. No functional changes.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging/imx: convert static vdev lists to list_head

Although not technically necessary because imx-media has only a
maximum of 8 video devices, and once setup the video device lists
are static, in anticipation of moving control ineritance to
v4l2-core, make the vdev lists more generic by converting to
dynamic list_head's.

After doing that, 'struct imx_media_pad' is now just a list_head
of video devices reachable from a pad. Allocate an array of list_head's,
one list_head for each pad, and attach that array to sd->host_priv.
An entry in the pad lists is of type 'struct imx_media_pad_vdev', and
points to a video device from the master list.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging/imx: remove static subdev arrays

For more complex OF graphs, there will be more async subdevices
registered. Remove the static subdev[IMX_MEDIA_MAX_SUBDEVS] array,
so that imx-media places no limits on the number of async subdevs
that can be added and registered.

There were two uses for 'struct imx_media_subdev'. First was to act
as the async subdev list to be passed to v4l2_async_notifier_register().

Second was to aid in inheriting subdev controls to the capture devices,
and this is done by creating a list of capture devices that can be reached
from a subdev's source pad. So 'struct imx_media_subdev' also contained
a static array of 'struct imx_media_pad' for placing the capture device
lists at each pad.

'struct imx_media_subdev' has been completely removed. Instead, at async
completion, allocate an array of 'struct imx_media_pad' and attach it to
the subdev's host_priv pointer, in order to support subdev controls
inheritance.

Likewise, remove static async_ptrs[IMX_MEDIA_MAX_SUBDEVS] array.
Instead, allocate a 'struct imx_media_async_subdev' when forming
the async list, and add it to an asd_list list_head in
imx_media_add_async_subdev(). At async completion, allocate the
asd pointer list and pull the asd's off asd_list for
v4l2_async_notifier_register().

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging/imx: pass fwnode handle to find/add async subdev

Pass the subdev's fwnode_handle to imx_media_find_async_subdev() and
imx_media_add_async_subdev(), instead of a device_node.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging/imx: remove devname string from imx_media_subdev

A separate string for the device name, for DEVNAME async match, was
never needed. Just assign the asd device name to the passed platform
device name pointer in imx_media_add_async_subdev().

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging/imx: of: allow for recursing downstream

Calling of_parse_subdev() recursively to a downstream path that has
already been followed is ok, it just means that call will return
immediately since the subdevice was already added to the async list.

With that there is no need to determine whether a subdevice's port
is a sink or source, so 'num_{sink|src}_pads' is no longer used and
is removed.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging/imx: remove static media link arrays

Remove the static list of media links that were formed at probe time.
These links can instead be created after all registered async subdevices
have been bound in imx_media_probe_complete().

The media links between subdevices that exist in the device tree, can
be created post-async completion by using v4l2_fwnode_parse_link() for
each endpoint node of that subdevice. Note this approach assumes
device-tree ports are equivalent to media pads (pad index equals
port id), and that device-tree endpoints are equivalent to media
links between pads.

Because links are no longer parsed by imx_media_of_parse(), its sole
function is now only to add subdevices that it encounters by walking
the OF graph to the async list, so the function has been renamed
imx_media_add_of_subdevs().

Similarly, the media links between the IPU-internal subdevice pads (the
CSI source pads, and all pads between the vdic, ic-prp, ic-prpenc, and
ic-prpvf subdevices), can be created post-async completion by looping
through the subdevice's media pads and using the const internal_subdev
table.

Because links are no longer parsed by imx_media_add_internal_subdevs(),
this function no longer needs an array of CSI subdevs to form links
from.

In summary, the following functions, which were used to form a list
of media links at probe time, are removed:

imx_media_add_pad_link()
add_internal_links()
of_add_pad_link()

replaced by these functions, called at probe time, which only populate
the async subdev list:

imx_media_add_of_subdevs()
imx_media_add_internal_subdevs()

and these functions, called at async completion, which create the
media links:

imx_media_create_of_links()
imx_media_create_csi_of_links()
imx_media_create_internal_links()

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging/imx: get CSI bus type from nearest upstream entity

The imx-media driver currently supports a device tree graph of
limited complexity. This patch is a first step in allowing imx-media
to work with more general OF graphs.

The CSI subdevice assumes the originating upstream subdevice (the
"sensor") is connected directly to either the CSI mux or the MIPI
CSI-2 receiver. But for more complex graphs, the sensor can be distant,
with possible bridge entities in between. Thus the sensor's bus type
could be quite different from what is entering the CSI. For example
a distant sensor could have a parallel interface, but the stream
entering the i.MX is MIPI CSI-2.

To remove this assumption, get the entering bus config from the entity
that is directly upstream from either the CSI mux, or the CSI-2 receiver.
If the CSI-2 receiver is not in the enabled pipeline, the bus type to the
CSI is parallel, otherwise the CSI is receiving MIPI CSI-2.

Note that we can't use the direct upstream source connected to CSI
(which is either the CSI mux or the CSI-2 receiver) to determine
bus type. The bus entering the CSI from the CSI-2 receiver is a 32-bit
parallel bus containing the demultiplexed MIPI CSI-2 virtual channels.
But the CSI and its IDMAC channels must be configured based on whether
it is receiving data from the CSI-2 receiver or from the CSI mux's
parallel interface pins.

The function csi_get_upstream_endpoint() is used to find this
endpoint. It makes use of a new utility function
imx_media_find_upstream_pad(), that if given a grp_id of 0, will
return the closest upstream pad from start_entity.

With these changes, imx_media_find_sensor() is no longer used and
is removed. As a result there is also no longer a need to identify
any sensor or set the sensor subdev's group id as a method to search
for it. So IMX_MEDIA_GRP_ID_SENSOR is removed. Also the video-mux group
id IMX_MEDIA_GRP_ID_VIDMUX was never used so that is removed as well.
The remaining IMX_MEDIA_GRP_ID_* definitions are entities internal
to the i.MX.

Another use of imx_media_find_sensor() in the CSI was to call the
sensor's g_skip_frames op to determine if a delay was needed before
enabling the CSI at stream on. If necessary this will have to be
re-addressed at a later time.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: staging: imx: use ktime_t for timestamps

The imx media driver passes around monotonic timestamps in the deprecated
'timespec' format. This is not a problem for the driver, as they won't
overflow, but moving to either timespec64 or ktime_t is preferred.

I'm picking ktime_t for simplicity here. frame_interval_monitor() is
the main function that changes, as it tries to compare a time interval
in microseconds. The algorithm slightly changes here, to avoid 64-bit
division. The code previously assumed that the error was at most 32-bit
worth of microseconds here, so I'm making the same assumption but add
an explicit test for it.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

media: Staging: media: imx: Prefer using BIT macro

Use BIT(x) instead of (1<<x).

Signed-off-by: Srishti Sharma <srishtishar@gmail.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

staging: imx: use ktime_t for timestamps

The imx media driver passes around monotonic timestamps in the deprecated
'timespec' format. This is not a problem for the driver, as they won't
overflow, but moving to either timespec64 or ktime_t is preferred.

I'm picking ktime_t for simplicity here. frame_interval_monitor() is
the main function that changes, as it tries to compare a time interval
in microseconds. The algorithm slightly changes here, to avoid 64-bit
division. The code previously assumed that the error was at most 32-bit
worth of microseconds here, so I'm making the same assumption but add
an explicit test for it.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

[media] media: imx: set and propagate default field, colorimetry

This patch adds a call to imx_media_fill_default_mbus_fields() in the
*_try_fmt() functions at the sink pads, to set empty field order and
colorimetry parameters.

If the field order is set to ANY, choose the currently set field order
at the sink pad. If the colorspace is set to DEFAULT, choose the
current colorspace at the sink pad. If any of xfer_func, ycbcr_enc
or quantization are set to DEFAULT, either choose the current sink pad
setting, or the default setting for the new colorspace, if non-DEFAULT
colorspace was given.

If a format is destined to be routed through the Image Converter,
fixed quantization and Y`CbCr encoding must be set.

Colorimetry is also propagated from sink to source pads anywhere
this has not already been done.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

[media] media: Add i.MX media core driver

Add the core media driver for i.MX SOC.

Switch from the v4l2_of_ APIs to the v4l2_fwnode_ APIs.
Add the bayer formats to imx-media's list of supported pixel and bus
formats.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>