1i.MX Video Capture Driver
2=========================
3
4Introduction
5------------
6
7The Freescale i.MX5/6 contains an Image Processing Unit (IPU), which
8handles the flow of image frames to and from capture devices and
9display devices.
10
11For image capture, the IPU contains the following internal subunits:
12
13- Image DMA Controller (IDMAC)
14- Camera Serial Interface (CSI)
15- Image Converter (IC)
16- Sensor Multi-FIFO Controller (SMFC)
17- Image Rotator (IRT)
18- Video De-Interlacing or Combining Block (VDIC)
19
20The IDMAC is the DMA controller for transfer of image frames to and from
21memory. Various dedicated DMA channels exist for both video capture and
22display paths. During transfer, the IDMAC is also capable of vertical
23image flip, 8x8 block transfer (see IRT description), pixel component
24re-ordering (for example UYVY to YUYV) within the same colorspace, and
25even packed <--> planar conversion. It can also perform a simple
26de-interlacing by interleaving even and odd lines during transfer
27(without motion compensation which requires the VDIC).
28
29The CSI is the backend capture unit that interfaces directly with
30camera sensors over Parallel, BT.656/1120, and MIPI CSI-2 busses.
31
32The IC handles color-space conversion, resizing (downscaling and
33upscaling), horizontal flip, and 90/270 degree rotation operations.
34
35There are three independent "tasks" within the IC that can carry out
36conversions concurrently: pre-process encoding, pre-process viewfinder,
37and post-processing. Within each task, conversions are split into three
38sections: downsizing section, main section (upsizing, flip, colorspace
39conversion, and graphics plane combining), and rotation section.
40
41The IPU time-shares the IC task operations. The time-slice granularity
42is one burst of eight pixels in the downsizing section, one image line
43in the main processing section, one image frame in the rotation section.
44
45The SMFC is composed of four independent FIFOs that each can transfer
46captured frames from sensors directly to memory concurrently via four
47IDMAC channels.
48
49The IRT carries out 90 and 270 degree image rotation operations. The
50rotation operation is carried out on 8x8 pixel blocks at a time. This
51operation is supported by the IDMAC which handles the 8x8 block transfer
52along with block reordering, in coordination with vertical flip.
53
54The VDIC handles the conversion of interlaced video to progressive, with
55support for different motion compensation modes (low, medium, and high
56motion). The deinterlaced output frames from the VDIC can be sent to the
57IC pre-process viewfinder task for further conversions. The VDIC also
58contains a Combiner that combines two image planes, with alpha blending
59and color keying.
60
61In addition to the IPU internal subunits, there are also two units
62outside the IPU that are also involved in video capture on i.MX:
63
64- MIPI CSI-2 Receiver for camera sensors with the MIPI CSI-2 bus
65  interface. This is a Synopsys DesignWare core.
66- Two video multiplexers for selecting among multiple sensor inputs
67  to send to a CSI.
68
69For more info, refer to the latest versions of the i.MX5/6 reference
70manuals [#f1]_ and [#f2]_.
71
72
73Features
74--------
75
76Some of the features of this driver include:
77
78- Many different pipelines can be configured via media controller API,
79  that correspond to the hardware video capture pipelines supported in
80  the i.MX.
81
82- Supports parallel, BT.565, and MIPI CSI-2 interfaces.
83
84- Concurrent independent streams, by configuring pipelines to multiple
85  video capture interfaces using independent entities.
86
87- Scaling, color-space conversion, horizontal and vertical flip, and
88  image rotation via IC task subdevs.
89
90- Many pixel formats supported (RGB, packed and planar YUV, partial
91  planar YUV).
92
93- The VDIC subdev supports motion compensated de-interlacing, with three
94  motion compensation modes: low, medium, and high motion. Pipelines are
95  defined that allow sending frames to the VDIC subdev directly from the
96  CSI. There is also support in the future for sending frames to the
97  VDIC from memory buffers via a output/mem2mem devices.
98
99- Includes a Frame Interval Monitor (FIM) that can correct vertical sync
100  problems with the ADV718x video decoders.
101
102
103Entities
104--------
105
106imx6-mipi-csi2
107--------------
108
109This is the MIPI CSI-2 receiver entity. It has one sink pad to receive
110the MIPI CSI-2 stream (usually from a MIPI CSI-2 camera sensor). It has
111four source pads, corresponding to the four MIPI CSI-2 demuxed virtual
112channel outputs. Multiple source pads can be enabled to independently
113stream from multiple virtual channels.
114
115This entity actually consists of two sub-blocks. One is the MIPI CSI-2
116core. This is a Synopsys Designware MIPI CSI-2 core. The other sub-block
117is a "CSI-2 to IPU gasket". The gasket acts as a demultiplexer of the
118four virtual channels streams, providing four separate parallel buses
119containing each virtual channel that are routed to CSIs or video
120multiplexers as described below.
121
122On i.MX6 solo/dual-lite, all four virtual channel buses are routed to
123two video multiplexers. Both CSI0 and CSI1 can receive any virtual
124channel, as selected by the video multiplexers.
125
126On i.MX6 Quad, virtual channel 0 is routed to IPU1-CSI0 (after selected
127by a video mux), virtual channels 1 and 2 are hard-wired to IPU1-CSI1
128and IPU2-CSI0, respectively, and virtual channel 3 is routed to
129IPU2-CSI1 (again selected by a video mux).
130
131ipuX_csiY_mux
132-------------
133
134These are the video multiplexers. They have two or more sink pads to
135select from either camera sensors with a parallel interface, or from
136MIPI CSI-2 virtual channels from imx6-mipi-csi2 entity. They have a
137single source pad that routes to a CSI (ipuX_csiY entities).
138
139On i.MX6 solo/dual-lite, there are two video mux entities. One sits
140in front of IPU1-CSI0 to select between a parallel sensor and any of
141the four MIPI CSI-2 virtual channels (a total of five sink pads). The
142other mux sits in front of IPU1-CSI1, and again has five sink pads to
143select between a parallel sensor and any of the four MIPI CSI-2 virtual
144channels.
145
146On i.MX6 Quad, there are two video mux entities. One sits in front of
147IPU1-CSI0 to select between a parallel sensor and MIPI CSI-2 virtual
148channel 0 (two sink pads). The other mux sits in front of IPU2-CSI1 to
149select between a parallel sensor and MIPI CSI-2 virtual channel 3 (two
150sink pads).
151
152ipuX_csiY
153---------
154
155These are the CSI entities. They have a single sink pad receiving from
156either a video mux or from a MIPI CSI-2 virtual channel as described
157above.
158
159This entity has two source pads. The first source pad can link directly
160to the ipuX_vdic entity or the ipuX_ic_prp entity, using hardware links
161that require no IDMAC memory buffer transfer.
162
163When the direct source pad is routed to the ipuX_ic_prp entity, frames
164from the CSI can be processed by one or both of the IC pre-processing
165tasks.
166
167When the direct source pad is routed to the ipuX_vdic entity, the VDIC
168will carry out motion-compensated de-interlace using "high motion" mode
169(see description of ipuX_vdic entity).
170
171The second source pad sends video frames directly to memory buffers
172via the SMFC and an IDMAC channel, bypassing IC pre-processing. This
173source pad is routed to a capture device node, with a node name of the
174format "ipuX_csiY capture".
175
176Note that since the IDMAC source pad makes use of an IDMAC channel, it
177can do pixel reordering within the same colorspace. For example, the
178sink pad can take UYVY2X8, but the IDMAC source pad can output YUYV2X8.
179If the sink pad is receiving YUV, the output at the capture device can
180also be converted to a planar YUV format such as YUV420.
181
182It will also perform simple de-interlace without motion compensation,
183which is activated if the sink pad's field type is an interlaced type,
184and the IDMAC source pad field type is set to none.
185
186This subdev can generate the following event when enabling the second
187IDMAC source pad:
188
189- V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR
190
191The user application can subscribe to this event from the ipuX_csiY
192subdev node. This event is generated by the Frame Interval Monitor
193(see below for more on the FIM).
194
195Cropping in ipuX_csiY
196---------------------
197
198The CSI supports cropping the incoming raw sensor frames. This is
199implemented in the ipuX_csiY entities at the sink pad, using the
200crop selection subdev API.
201
202The CSI also supports fixed divide-by-two downscaling indepently in
203width and height. This is implemented in the ipuX_csiY entities at
204the sink pad, using the compose selection subdev API.
205
206The output rectangle at the ipuX_csiY source pad is the same as
207the compose rectangle at the sink pad. So the source pad rectangle
208cannot be negotiated, it must be set using the compose selection
209API at sink pad (if /2 downscale is desired, otherwise source pad
210rectangle is equal to incoming rectangle).
211
212To give an example of crop and /2 downscale, this will crop a
2131280x960 input frame to 640x480, and then /2 downscale in both
214dimensions to 320x240 (assumes ipu1_csi0 is linked to ipu1_csi0_mux):
215
216.. code-block:: none
217
218   media-ctl -V "'ipu1_csi0_mux':2[fmt:UYVY2X8/1280x960]"
219   media-ctl -V "'ipu1_csi0':0[crop:(0,0)/640x480]"
220   media-ctl -V "'ipu1_csi0':0[compose:(0,0)/320x240]"
221
222Frame Skipping in ipuX_csiY
223---------------------------
224
225The CSI supports frame rate decimation, via frame skipping. Frame
226rate decimation is specified by setting the frame intervals at
227sink and source pads. The ipuX_csiY entity then applies the best
228frame skip setting to the CSI to achieve the desired frame rate
229at the source pad.
230
231The following example reduces an assumed incoming 60 Hz frame
232rate by half at the IDMAC output source pad:
233
234.. code-block:: none
235
236   media-ctl -V "'ipu1_csi0':0[fmt:UYVY2X8/640x480@1/60]"
237   media-ctl -V "'ipu1_csi0':2[fmt:UYVY2X8/640x480@1/30]"
238
239Frame Interval Monitor in ipuX_csiY
240-----------------------------------
241
242The adv718x decoders can occasionally send corrupt fields during
243NTSC/PAL signal re-sync (too little or too many video lines). When
244this happens, the IPU triggers a mechanism to re-establish vertical
245sync by adding 1 dummy line every frame, which causes a rolling effect
246from image to image, and can last a long time before a stable image is
247recovered. Or sometimes the mechanism doesn't work at all, causing a
248permanent split image (one frame contains lines from two consecutive
249captured images).
250
251From experiment it was found that during image rolling, the frame
252intervals (elapsed time between two EOF's) drop below the nominal
253value for the current standard, by about one frame time (60 usec),
254and remain at that value until rolling stops.
255
256While the reason for this observation isn't known (the IPU dummy
257line mechanism should show an increase in the intervals by 1 line
258time every frame, not a fixed value), we can use it to detect the
259corrupt fields using a frame interval monitor. If the FIM detects a
260bad frame interval, the ipuX_csiY subdev will send the event
261V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR. Userland can register with
262the FIM event notification on the ipuX_csiY subdev device node.
263Userland can issue a streaming restart when this event is received
264to correct the rolling/split image.
265
266The ipuX_csiY subdev includes custom controls to tweak some dials for
267FIM. If one of these controls is changed during streaming, the FIM will
268be reset and will continue at the new settings.
269
270- V4L2_CID_IMX_FIM_ENABLE
271
272Enable/disable the FIM.
273
274- V4L2_CID_IMX_FIM_NUM
275
276How many frame interval measurements to average before comparing against
277the nominal frame interval reported by the sensor. This can reduce noise
278caused by interrupt latency.
279
280- V4L2_CID_IMX_FIM_TOLERANCE_MIN
281
282If the averaged intervals fall outside nominal by this amount, in
283microseconds, the V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR event is sent.
284
285- V4L2_CID_IMX_FIM_TOLERANCE_MAX
286
287If any intervals are higher than this value, those samples are
288discarded and do not enter into the average. This can be used to
289discard really high interval errors that might be due to interrupt
290latency from high system load.
291
292- V4L2_CID_IMX_FIM_NUM_SKIP
293
294How many frames to skip after a FIM reset or stream restart before
295FIM begins to average intervals.
296
297- V4L2_CID_IMX_FIM_ICAP_CHANNEL
298- V4L2_CID_IMX_FIM_ICAP_EDGE
299
300These controls will configure an input capture channel as the method
301for measuring frame intervals. This is superior to the default method
302of measuring frame intervals via EOF interrupt, since it is not subject
303to uncertainty errors introduced by interrupt latency.
304
305Input capture requires hardware support. A VSYNC signal must be routed
306to one of the i.MX6 input capture channel pads.
307
308V4L2_CID_IMX_FIM_ICAP_CHANNEL configures which i.MX6 input capture
309channel to use. This must be 0 or 1.
310
311V4L2_CID_IMX_FIM_ICAP_EDGE configures which signal edge will trigger
312input capture events. By default the input capture method is disabled
313with a value of IRQ_TYPE_NONE. Set this control to IRQ_TYPE_EDGE_RISING,
314IRQ_TYPE_EDGE_FALLING, or IRQ_TYPE_EDGE_BOTH to enable input capture,
315triggered on the given signal edge(s).
316
317When input capture is disabled, frame intervals will be measured via
318EOF interrupt.
319
320
321ipuX_vdic
322---------
323
324The VDIC carries out motion compensated de-interlacing, with three
325motion compensation modes: low, medium, and high motion. The mode is
326specified with the menu control V4L2_CID_DEINTERLACING_MODE. It has
327two sink pads and a single source pad.
328
329The direct sink pad receives from an ipuX_csiY direct pad. With this
330link the VDIC can only operate in high motion mode.
331
332When the IDMAC sink pad is activated, it receives from an output
333or mem2mem device node. With this pipeline, it can also operate
334in low and medium modes, because these modes require receiving
335frames from memory buffers. Note that an output or mem2mem device
336is not implemented yet, so this sink pad currently has no links.
337
338The source pad routes to the IC pre-processing entity ipuX_ic_prp.
339
340ipuX_ic_prp
341-----------
342
343This is the IC pre-processing entity. It acts as a router, routing
344data from its sink pad to one or both of its source pads.
345
346It has a single sink pad. The sink pad can receive from the ipuX_csiY
347direct pad, or from ipuX_vdic.
348
349This entity has two source pads. One source pad routes to the
350pre-process encode task entity (ipuX_ic_prpenc), the other to the
351pre-process viewfinder task entity (ipuX_ic_prpvf). Both source pads
352can be activated at the same time if the sink pad is receiving from
353ipuX_csiY. Only the source pad to the pre-process viewfinder task entity
354can be activated if the sink pad is receiving from ipuX_vdic (frames
355from the VDIC can only be processed by the pre-process viewfinder task).
356
357ipuX_ic_prpenc
358--------------
359
360This is the IC pre-processing encode entity. It has a single sink
361pad from ipuX_ic_prp, and a single source pad. The source pad is
362routed to a capture device node, with a node name of the format
363"ipuX_ic_prpenc capture".
364
365This entity performs the IC pre-process encode task operations:
366color-space conversion, resizing (downscaling and upscaling),
367horizontal and vertical flip, and 90/270 degree rotation. Flip
368and rotation are provided via standard V4L2 controls.
369
370Like the ipuX_csiY IDMAC source, it can also perform simple de-interlace
371without motion compensation, and pixel reordering.
372
373ipuX_ic_prpvf
374-------------
375
376This is the IC pre-processing viewfinder entity. It has a single sink
377pad from ipuX_ic_prp, and a single source pad. The source pad is routed
378to a capture device node, with a node name of the format
379"ipuX_ic_prpvf capture".
380
381It is identical in operation to ipuX_ic_prpenc, with the same resizing
382and CSC operations and flip/rotation controls. It will receive and
383process de-interlaced frames from the ipuX_vdic if ipuX_ic_prp is
384receiving from ipuX_vdic.
385
386Like the ipuX_csiY IDMAC source, it can perform simple de-interlace
387without motion compensation. However, note that if the ipuX_vdic is
388included in the pipeline (ipuX_ic_prp is receiving from ipuX_vdic),
389it's not possible to use simple de-interlace in ipuX_ic_prpvf, since
390the ipuX_vdic has already carried out de-interlacing (with motion
391compensation) and therefore the field type output from ipuX_ic_prp can
392only be none.
393
394Capture Pipelines
395-----------------
396
397The following describe the various use-cases supported by the pipelines.
398
399The links shown do not include the backend sensor, video mux, or mipi
400csi-2 receiver links. This depends on the type of sensor interface
401(parallel or mipi csi-2). So these pipelines begin with:
402
403sensor -> ipuX_csiY_mux -> ...
404
405for parallel sensors, or:
406
407sensor -> imx6-mipi-csi2 -> (ipuX_csiY_mux) -> ...
408
409for mipi csi-2 sensors. The imx6-mipi-csi2 receiver may need to route
410to the video mux (ipuX_csiY_mux) before sending to the CSI, depending
411on the mipi csi-2 virtual channel, hence ipuX_csiY_mux is shown in
412parenthesis.
413
414Unprocessed Video Capture:
415--------------------------
416
417Send frames directly from sensor to camera device interface node, with
418no conversions, via ipuX_csiY IDMAC source pad:
419
420-> ipuX_csiY:2 -> ipuX_csiY capture
421
422IC Direct Conversions:
423----------------------
424
425This pipeline uses the preprocess encode entity to route frames directly
426from the CSI to the IC, to carry out scaling up to 1024x1024 resolution,
427CSC, flipping, and image rotation:
428
429-> ipuX_csiY:1 -> 0:ipuX_ic_prp:1 -> 0:ipuX_ic_prpenc:1 -> ipuX_ic_prpenc capture
430
431Motion Compensated De-interlace:
432--------------------------------
433
434This pipeline routes frames from the CSI direct pad to the VDIC entity to
435support motion-compensated de-interlacing (high motion mode only),
436scaling up to 1024x1024, CSC, flip, and rotation:
437
438-> ipuX_csiY:1 -> 0:ipuX_vdic:2 -> 0:ipuX_ic_prp:2 -> 0:ipuX_ic_prpvf:1 -> ipuX_ic_prpvf capture
439
440
441Usage Notes
442-----------
443
444To aid in configuration and for backward compatibility with V4L2
445applications that access controls only from video device nodes, the
446capture device interfaces inherit controls from the active entities
447in the current pipeline, so controls can be accessed either directly
448from the subdev or from the active capture device interface. For
449example, the FIM controls are available either from the ipuX_csiY
450subdevs or from the active capture device.
451
452The following are specific usage notes for the Sabre* reference
453boards:
454
455
456SabreLite with OV5642 and OV5640
457--------------------------------
458
459This platform requires the OmniVision OV5642 module with a parallel
460camera interface, and the OV5640 module with a MIPI CSI-2
461interface. Both modules are available from Boundary Devices:
462
463- https://boundarydevices.com/product/nit6x_5mp
464- https://boundarydevices.com/product/nit6x_5mp_mipi
465
466Note that if only one camera module is available, the other sensor
467node can be disabled in the device tree.
468
469The OV5642 module is connected to the parallel bus input on the i.MX
470internal video mux to IPU1 CSI0. It's i2c bus connects to i2c bus 2.
471
472The MIPI CSI-2 OV5640 module is connected to the i.MX internal MIPI CSI-2
473receiver, and the four virtual channel outputs from the receiver are
474routed as follows: vc0 to the IPU1 CSI0 mux, vc1 directly to IPU1 CSI1,
475vc2 directly to IPU2 CSI0, and vc3 to the IPU2 CSI1 mux. The OV5640 is
476also connected to i2c bus 2 on the SabreLite, therefore the OV5642 and
477OV5640 must not share the same i2c slave address.
478
479The following basic example configures unprocessed video capture
480pipelines for both sensors. The OV5642 is routed to ipu1_csi0, and
481the OV5640, transmitting on MIPI CSI-2 virtual channel 1 (which is
482imx6-mipi-csi2 pad 2), is routed to ipu1_csi1. Both sensors are
483configured to output 640x480, and the OV5642 outputs YUYV2X8, the
484OV5640 UYVY2X8:
485
486.. code-block:: none
487
488   # Setup links for OV5642
489   media-ctl -l "'ov5642 1-0042':0 -> 'ipu1_csi0_mux':1[1]"
490   media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
491   media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
492   # Setup links for OV5640
493   media-ctl -l "'ov5640 1-0040':0 -> 'imx6-mipi-csi2':0[1]"
494   media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
495   media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
496   # Configure pads for OV5642 pipeline
497   media-ctl -V "'ov5642 1-0042':0 [fmt:YUYV2X8/640x480 field:none]"
498   media-ctl -V "'ipu1_csi0_mux':2 [fmt:YUYV2X8/640x480 field:none]"
499   media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/640x480 field:none]"
500   # Configure pads for OV5640 pipeline
501   media-ctl -V "'ov5640 1-0040':0 [fmt:UYVY2X8/640x480 field:none]"
502   media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/640x480 field:none]"
503   media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/640x480 field:none]"
504
505Streaming can then begin independently on the capture device nodes
506"ipu1_csi0 capture" and "ipu1_csi1 capture". The v4l2-ctl tool can
507be used to select any supported YUV pixelformat on the capture device
508nodes, including planar.
509
510SabreAuto with ADV7180 decoder
511------------------------------
512
513On the SabreAuto, an on-board ADV7180 SD decoder is connected to the
514parallel bus input on the internal video mux to IPU1 CSI0.
515
516The following example configures a pipeline to capture from the ADV7180
517video decoder, assuming NTSC 720x480 input signals, with Motion
518Compensated de-interlacing. Pad field types assume the adv7180 outputs
519"interlaced". $outputfmt can be any format supported by the ipu1_ic_prpvf
520entity at its output pad:
521
522.. code-block:: none
523
524   # Setup links
525   media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]"
526   media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
527   media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]"
528   media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]"
529   media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]"
530   media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]"
531   # Configure pads
532   media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480]"
533   media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480 field:interlaced]"
534   media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480 field:interlaced]"
535   media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]"
536   media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]"
537   media-ctl -V "'ipu1_ic_prpvf':1 [fmt:$outputfmt field:none]"
538
539Streaming can then begin on the capture device node at
540"ipu1_ic_prpvf capture". The v4l2-ctl tool can be used to select any
541supported YUV or RGB pixelformat on the capture device node.
542
543This platform accepts Composite Video analog inputs to the ADV7180 on
544Ain1 (connector J42).
545
546SabreSD with MIPI CSI-2 OV5640
547------------------------------
548
549Similarly to SabreLite, the SabreSD supports a parallel interface
550OV5642 module on IPU1 CSI0, and a MIPI CSI-2 OV5640 module. The OV5642
551connects to i2c bus 1 and the OV5640 to i2c bus 2.
552
553The device tree for SabreSD includes OF graphs for both the parallel
554OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI
555CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled.
556The OV5640 module connects to MIPI connector J5 (sorry I don't have the
557compatible module part number or URL).
558
559The following example configures a direct conversion pipeline to capture
560from the OV5640, transmitting on MIPI CSI-2 virtual channel 1. $sensorfmt
561can be any format supported by the OV5640. $sensordim is the frame
562dimension part of $sensorfmt (minus the mbus pixel code). $outputfmt can
563be any format supported by the ipu1_ic_prpenc entity at its output pad:
564
565.. code-block:: none
566
567   # Setup links
568   media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
569   media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
570   media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]"
571   media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
572   media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
573   # Configure pads
574   media-ctl -V "'ov5640 1-003c':0 [fmt:$sensorfmt field:none]"
575   media-ctl -V "'imx6-mipi-csi2':2 [fmt:$sensorfmt field:none]"
576   media-ctl -V "'ipu1_csi1':1 [fmt:AYUV32/$sensordim field:none]"
577   media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/$sensordim field:none]"
578   media-ctl -V "'ipu1_ic_prpenc':1 [fmt:$outputfmt field:none]"
579
580Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
581tool can be used to select any supported YUV or RGB pixelformat on the
582capture device node.
583
584
585Known Issues
586------------
587
5881. When using 90 or 270 degree rotation control at capture resolutions
589   near the IC resizer limit of 1024x1024, and combined with planar
590   pixel formats (YUV420, YUV422p), frame capture will often fail with
591   no end-of-frame interrupts from the IDMAC channel. To work around
592   this, use lower resolution and/or packed formats (YUYV, RGB3, etc.)
593   when 90 or 270 rotations are needed.
594
595
596File list
597---------
598
599drivers/staging/media/imx/
600include/media/imx.h
601include/linux/imx-media.h
602
603References
604----------
605
606.. [#f1] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6DQRM.pdf
607.. [#f2] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6SDLRM.pdf
608
609
610Authors
611-------
612
613- Steve Longerbeam <steve_longerbeam@mentor.com>
614- Philipp Zabel <kernel@pengutronix.de>
615- Russell King <linux@armlinux.org.uk>
616
617Copyright (C) 2012-2017 Mentor Graphics Inc.
618