1i.MX Video Capture Driver 2========================= 3 4Introduction 5------------ 6 7The Freescale i.MX5/6 contains an Image Processing Unit (IPU), which 8handles the flow of image frames to and from capture devices and 9display devices. 10 11For image capture, the IPU contains the following internal subunits: 12 13- Image DMA Controller (IDMAC) 14- Camera Serial Interface (CSI) 15- Image Converter (IC) 16- Sensor Multi-FIFO Controller (SMFC) 17- Image Rotator (IRT) 18- Video De-Interlacing or Combining Block (VDIC) 19 20The IDMAC is the DMA controller for transfer of image frames to and from 21memory. Various dedicated DMA channels exist for both video capture and 22display paths. During transfer, the IDMAC is also capable of vertical 23image flip, 8x8 block transfer (see IRT description), pixel component 24re-ordering (for example UYVY to YUYV) within the same colorspace, and 25even packed <--> planar conversion. It can also perform a simple 26de-interlacing by interleaving even and odd lines during transfer 27(without motion compensation which requires the VDIC). 28 29The CSI is the backend capture unit that interfaces directly with 30camera sensors over Parallel, BT.656/1120, and MIPI CSI-2 busses. 31 32The IC handles color-space conversion, resizing (downscaling and 33upscaling), horizontal flip, and 90/270 degree rotation operations. 34 35There are three independent "tasks" within the IC that can carry out 36conversions concurrently: pre-process encoding, pre-process viewfinder, 37and post-processing. Within each task, conversions are split into three 38sections: downsizing section, main section (upsizing, flip, colorspace 39conversion, and graphics plane combining), and rotation section. 40 41The IPU time-shares the IC task operations. The time-slice granularity 42is one burst of eight pixels in the downsizing section, one image line 43in the main processing section, one image frame in the rotation section. 44 45The SMFC is composed of four independent FIFOs that each can transfer 46captured frames from sensors directly to memory concurrently via four 47IDMAC channels. 48 49The IRT carries out 90 and 270 degree image rotation operations. The 50rotation operation is carried out on 8x8 pixel blocks at a time. This 51operation is supported by the IDMAC which handles the 8x8 block transfer 52along with block reordering, in coordination with vertical flip. 53 54The VDIC handles the conversion of interlaced video to progressive, with 55support for different motion compensation modes (low, medium, and high 56motion). The deinterlaced output frames from the VDIC can be sent to the 57IC pre-process viewfinder task for further conversions. The VDIC also 58contains a Combiner that combines two image planes, with alpha blending 59and color keying. 60 61In addition to the IPU internal subunits, there are also two units 62outside the IPU that are also involved in video capture on i.MX: 63 64- MIPI CSI-2 Receiver for camera sensors with the MIPI CSI-2 bus 65 interface. This is a Synopsys DesignWare core. 66- Two video multiplexers for selecting among multiple sensor inputs 67 to send to a CSI. 68 69For more info, refer to the latest versions of the i.MX5/6 reference 70manuals [#f1]_ and [#f2]_. 71 72 73Features 74-------- 75 76Some of the features of this driver include: 77 78- Many different pipelines can be configured via media controller API, 79 that correspond to the hardware video capture pipelines supported in 80 the i.MX. 81 82- Supports parallel, BT.565, and MIPI CSI-2 interfaces. 83 84- Concurrent independent streams, by configuring pipelines to multiple 85 video capture interfaces using independent entities. 86 87- Scaling, color-space conversion, horizontal and vertical flip, and 88 image rotation via IC task subdevs. 89 90- Many pixel formats supported (RGB, packed and planar YUV, partial 91 planar YUV). 92 93- The VDIC subdev supports motion compensated de-interlacing, with three 94 motion compensation modes: low, medium, and high motion. Pipelines are 95 defined that allow sending frames to the VDIC subdev directly from the 96 CSI. There is also support in the future for sending frames to the 97 VDIC from memory buffers via a output/mem2mem devices. 98 99- Includes a Frame Interval Monitor (FIM) that can correct vertical sync 100 problems with the ADV718x video decoders. 101 102 103Entities 104-------- 105 106imx6-mipi-csi2 107-------------- 108 109This is the MIPI CSI-2 receiver entity. It has one sink pad to receive 110the MIPI CSI-2 stream (usually from a MIPI CSI-2 camera sensor). It has 111four source pads, corresponding to the four MIPI CSI-2 demuxed virtual 112channel outputs. Multiple source pads can be enabled to independently 113stream from multiple virtual channels. 114 115This entity actually consists of two sub-blocks. One is the MIPI CSI-2 116core. This is a Synopsys Designware MIPI CSI-2 core. The other sub-block 117is a "CSI-2 to IPU gasket". The gasket acts as a demultiplexer of the 118four virtual channels streams, providing four separate parallel buses 119containing each virtual channel that are routed to CSIs or video 120multiplexers as described below. 121 122On i.MX6 solo/dual-lite, all four virtual channel buses are routed to 123two video multiplexers. Both CSI0 and CSI1 can receive any virtual 124channel, as selected by the video multiplexers. 125 126On i.MX6 Quad, virtual channel 0 is routed to IPU1-CSI0 (after selected 127by a video mux), virtual channels 1 and 2 are hard-wired to IPU1-CSI1 128and IPU2-CSI0, respectively, and virtual channel 3 is routed to 129IPU2-CSI1 (again selected by a video mux). 130 131ipuX_csiY_mux 132------------- 133 134These are the video multiplexers. They have two or more sink pads to 135select from either camera sensors with a parallel interface, or from 136MIPI CSI-2 virtual channels from imx6-mipi-csi2 entity. They have a 137single source pad that routes to a CSI (ipuX_csiY entities). 138 139On i.MX6 solo/dual-lite, there are two video mux entities. One sits 140in front of IPU1-CSI0 to select between a parallel sensor and any of 141the four MIPI CSI-2 virtual channels (a total of five sink pads). The 142other mux sits in front of IPU1-CSI1, and again has five sink pads to 143select between a parallel sensor and any of the four MIPI CSI-2 virtual 144channels. 145 146On i.MX6 Quad, there are two video mux entities. One sits in front of 147IPU1-CSI0 to select between a parallel sensor and MIPI CSI-2 virtual 148channel 0 (two sink pads). The other mux sits in front of IPU2-CSI1 to 149select between a parallel sensor and MIPI CSI-2 virtual channel 3 (two 150sink pads). 151 152ipuX_csiY 153--------- 154 155These are the CSI entities. They have a single sink pad receiving from 156either a video mux or from a MIPI CSI-2 virtual channel as described 157above. 158 159This entity has two source pads. The first source pad can link directly 160to the ipuX_vdic entity or the ipuX_ic_prp entity, using hardware links 161that require no IDMAC memory buffer transfer. 162 163When the direct source pad is routed to the ipuX_ic_prp entity, frames 164from the CSI can be processed by one or both of the IC pre-processing 165tasks. 166 167When the direct source pad is routed to the ipuX_vdic entity, the VDIC 168will carry out motion-compensated de-interlace using "high motion" mode 169(see description of ipuX_vdic entity). 170 171The second source pad sends video frames directly to memory buffers 172via the SMFC and an IDMAC channel, bypassing IC pre-processing. This 173source pad is routed to a capture device node, with a node name of the 174format "ipuX_csiY capture". 175 176Note that since the IDMAC source pad makes use of an IDMAC channel, it 177can do pixel reordering within the same colorspace. For example, the 178sink pad can take UYVY2X8, but the IDMAC source pad can output YUYV2X8. 179If the sink pad is receiving YUV, the output at the capture device can 180also be converted to a planar YUV format such as YUV420. 181 182It will also perform simple de-interlace without motion compensation, 183which is activated if the sink pad's field type is an interlaced type, 184and the IDMAC source pad field type is set to none. 185 186This subdev can generate the following event when enabling the second 187IDMAC source pad: 188 189- V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR 190 191The user application can subscribe to this event from the ipuX_csiY 192subdev node. This event is generated by the Frame Interval Monitor 193(see below for more on the FIM). 194 195Cropping in ipuX_csiY 196--------------------- 197 198The CSI supports cropping the incoming raw sensor frames. This is 199implemented in the ipuX_csiY entities at the sink pad, using the 200crop selection subdev API. 201 202The CSI also supports fixed divide-by-two downscaling indepently in 203width and height. This is implemented in the ipuX_csiY entities at 204the sink pad, using the compose selection subdev API. 205 206The output rectangle at the ipuX_csiY source pad is the same as 207the compose rectangle at the sink pad. So the source pad rectangle 208cannot be negotiated, it must be set using the compose selection 209API at sink pad (if /2 downscale is desired, otherwise source pad 210rectangle is equal to incoming rectangle). 211 212To give an example of crop and /2 downscale, this will crop a 2131280x960 input frame to 640x480, and then /2 downscale in both 214dimensions to 320x240 (assumes ipu1_csi0 is linked to ipu1_csi0_mux): 215 216.. code-block:: none 217 218 media-ctl -V "'ipu1_csi0_mux':2[fmt:UYVY2X8/1280x960]" 219 media-ctl -V "'ipu1_csi0':0[crop:(0,0)/640x480]" 220 media-ctl -V "'ipu1_csi0':0[compose:(0,0)/320x240]" 221 222Frame Skipping in ipuX_csiY 223--------------------------- 224 225The CSI supports frame rate decimation, via frame skipping. Frame 226rate decimation is specified by setting the frame intervals at 227sink and source pads. The ipuX_csiY entity then applies the best 228frame skip setting to the CSI to achieve the desired frame rate 229at the source pad. 230 231The following example reduces an assumed incoming 60 Hz frame 232rate by half at the IDMAC output source pad: 233 234.. code-block:: none 235 236 media-ctl -V "'ipu1_csi0':0[fmt:UYVY2X8/640x480@1/60]" 237 media-ctl -V "'ipu1_csi0':2[fmt:UYVY2X8/640x480@1/30]" 238 239Frame Interval Monitor in ipuX_csiY 240----------------------------------- 241 242The adv718x decoders can occasionally send corrupt fields during 243NTSC/PAL signal re-sync (too little or too many video lines). When 244this happens, the IPU triggers a mechanism to re-establish vertical 245sync by adding 1 dummy line every frame, which causes a rolling effect 246from image to image, and can last a long time before a stable image is 247recovered. Or sometimes the mechanism doesn't work at all, causing a 248permanent split image (one frame contains lines from two consecutive 249captured images). 250 251From experiment it was found that during image rolling, the frame 252intervals (elapsed time between two EOF's) drop below the nominal 253value for the current standard, by about one frame time (60 usec), 254and remain at that value until rolling stops. 255 256While the reason for this observation isn't known (the IPU dummy 257line mechanism should show an increase in the intervals by 1 line 258time every frame, not a fixed value), we can use it to detect the 259corrupt fields using a frame interval monitor. If the FIM detects a 260bad frame interval, the ipuX_csiY subdev will send the event 261V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR. Userland can register with 262the FIM event notification on the ipuX_csiY subdev device node. 263Userland can issue a streaming restart when this event is received 264to correct the rolling/split image. 265 266The ipuX_csiY subdev includes custom controls to tweak some dials for 267FIM. If one of these controls is changed during streaming, the FIM will 268be reset and will continue at the new settings. 269 270- V4L2_CID_IMX_FIM_ENABLE 271 272Enable/disable the FIM. 273 274- V4L2_CID_IMX_FIM_NUM 275 276How many frame interval measurements to average before comparing against 277the nominal frame interval reported by the sensor. This can reduce noise 278caused by interrupt latency. 279 280- V4L2_CID_IMX_FIM_TOLERANCE_MIN 281 282If the averaged intervals fall outside nominal by this amount, in 283microseconds, the V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR event is sent. 284 285- V4L2_CID_IMX_FIM_TOLERANCE_MAX 286 287If any intervals are higher than this value, those samples are 288discarded and do not enter into the average. This can be used to 289discard really high interval errors that might be due to interrupt 290latency from high system load. 291 292- V4L2_CID_IMX_FIM_NUM_SKIP 293 294How many frames to skip after a FIM reset or stream restart before 295FIM begins to average intervals. 296 297- V4L2_CID_IMX_FIM_ICAP_CHANNEL 298- V4L2_CID_IMX_FIM_ICAP_EDGE 299 300These controls will configure an input capture channel as the method 301for measuring frame intervals. This is superior to the default method 302of measuring frame intervals via EOF interrupt, since it is not subject 303to uncertainty errors introduced by interrupt latency. 304 305Input capture requires hardware support. A VSYNC signal must be routed 306to one of the i.MX6 input capture channel pads. 307 308V4L2_CID_IMX_FIM_ICAP_CHANNEL configures which i.MX6 input capture 309channel to use. This must be 0 or 1. 310 311V4L2_CID_IMX_FIM_ICAP_EDGE configures which signal edge will trigger 312input capture events. By default the input capture method is disabled 313with a value of IRQ_TYPE_NONE. Set this control to IRQ_TYPE_EDGE_RISING, 314IRQ_TYPE_EDGE_FALLING, or IRQ_TYPE_EDGE_BOTH to enable input capture, 315triggered on the given signal edge(s). 316 317When input capture is disabled, frame intervals will be measured via 318EOF interrupt. 319 320 321ipuX_vdic 322--------- 323 324The VDIC carries out motion compensated de-interlacing, with three 325motion compensation modes: low, medium, and high motion. The mode is 326specified with the menu control V4L2_CID_DEINTERLACING_MODE. It has 327two sink pads and a single source pad. 328 329The direct sink pad receives from an ipuX_csiY direct pad. With this 330link the VDIC can only operate in high motion mode. 331 332When the IDMAC sink pad is activated, it receives from an output 333or mem2mem device node. With this pipeline, it can also operate 334in low and medium modes, because these modes require receiving 335frames from memory buffers. Note that an output or mem2mem device 336is not implemented yet, so this sink pad currently has no links. 337 338The source pad routes to the IC pre-processing entity ipuX_ic_prp. 339 340ipuX_ic_prp 341----------- 342 343This is the IC pre-processing entity. It acts as a router, routing 344data from its sink pad to one or both of its source pads. 345 346It has a single sink pad. The sink pad can receive from the ipuX_csiY 347direct pad, or from ipuX_vdic. 348 349This entity has two source pads. One source pad routes to the 350pre-process encode task entity (ipuX_ic_prpenc), the other to the 351pre-process viewfinder task entity (ipuX_ic_prpvf). Both source pads 352can be activated at the same time if the sink pad is receiving from 353ipuX_csiY. Only the source pad to the pre-process viewfinder task entity 354can be activated if the sink pad is receiving from ipuX_vdic (frames 355from the VDIC can only be processed by the pre-process viewfinder task). 356 357ipuX_ic_prpenc 358-------------- 359 360This is the IC pre-processing encode entity. It has a single sink 361pad from ipuX_ic_prp, and a single source pad. The source pad is 362routed to a capture device node, with a node name of the format 363"ipuX_ic_prpenc capture". 364 365This entity performs the IC pre-process encode task operations: 366color-space conversion, resizing (downscaling and upscaling), 367horizontal and vertical flip, and 90/270 degree rotation. Flip 368and rotation are provided via standard V4L2 controls. 369 370Like the ipuX_csiY IDMAC source, it can also perform simple de-interlace 371without motion compensation, and pixel reordering. 372 373ipuX_ic_prpvf 374------------- 375 376This is the IC pre-processing viewfinder entity. It has a single sink 377pad from ipuX_ic_prp, and a single source pad. The source pad is routed 378to a capture device node, with a node name of the format 379"ipuX_ic_prpvf capture". 380 381It is identical in operation to ipuX_ic_prpenc, with the same resizing 382and CSC operations and flip/rotation controls. It will receive and 383process de-interlaced frames from the ipuX_vdic if ipuX_ic_prp is 384receiving from ipuX_vdic. 385 386Like the ipuX_csiY IDMAC source, it can perform simple de-interlace 387without motion compensation. However, note that if the ipuX_vdic is 388included in the pipeline (ipuX_ic_prp is receiving from ipuX_vdic), 389it's not possible to use simple de-interlace in ipuX_ic_prpvf, since 390the ipuX_vdic has already carried out de-interlacing (with motion 391compensation) and therefore the field type output from ipuX_ic_prp can 392only be none. 393 394Capture Pipelines 395----------------- 396 397The following describe the various use-cases supported by the pipelines. 398 399The links shown do not include the backend sensor, video mux, or mipi 400csi-2 receiver links. This depends on the type of sensor interface 401(parallel or mipi csi-2). So these pipelines begin with: 402 403sensor -> ipuX_csiY_mux -> ... 404 405for parallel sensors, or: 406 407sensor -> imx6-mipi-csi2 -> (ipuX_csiY_mux) -> ... 408 409for mipi csi-2 sensors. The imx6-mipi-csi2 receiver may need to route 410to the video mux (ipuX_csiY_mux) before sending to the CSI, depending 411on the mipi csi-2 virtual channel, hence ipuX_csiY_mux is shown in 412parenthesis. 413 414Unprocessed Video Capture: 415-------------------------- 416 417Send frames directly from sensor to camera device interface node, with 418no conversions, via ipuX_csiY IDMAC source pad: 419 420-> ipuX_csiY:2 -> ipuX_csiY capture 421 422IC Direct Conversions: 423---------------------- 424 425This pipeline uses the preprocess encode entity to route frames directly 426from the CSI to the IC, to carry out scaling up to 1024x1024 resolution, 427CSC, flipping, and image rotation: 428 429-> ipuX_csiY:1 -> 0:ipuX_ic_prp:1 -> 0:ipuX_ic_prpenc:1 -> ipuX_ic_prpenc capture 430 431Motion Compensated De-interlace: 432-------------------------------- 433 434This pipeline routes frames from the CSI direct pad to the VDIC entity to 435support motion-compensated de-interlacing (high motion mode only), 436scaling up to 1024x1024, CSC, flip, and rotation: 437 438-> ipuX_csiY:1 -> 0:ipuX_vdic:2 -> 0:ipuX_ic_prp:2 -> 0:ipuX_ic_prpvf:1 -> ipuX_ic_prpvf capture 439 440 441Usage Notes 442----------- 443 444To aid in configuration and for backward compatibility with V4L2 445applications that access controls only from video device nodes, the 446capture device interfaces inherit controls from the active entities 447in the current pipeline, so controls can be accessed either directly 448from the subdev or from the active capture device interface. For 449example, the FIM controls are available either from the ipuX_csiY 450subdevs or from the active capture device. 451 452The following are specific usage notes for the Sabre* reference 453boards: 454 455 456SabreLite with OV5642 and OV5640 457-------------------------------- 458 459This platform requires the OmniVision OV5642 module with a parallel 460camera interface, and the OV5640 module with a MIPI CSI-2 461interface. Both modules are available from Boundary Devices: 462 463- https://boundarydevices.com/product/nit6x_5mp 464- https://boundarydevices.com/product/nit6x_5mp_mipi 465 466Note that if only one camera module is available, the other sensor 467node can be disabled in the device tree. 468 469The OV5642 module is connected to the parallel bus input on the i.MX 470internal video mux to IPU1 CSI0. It's i2c bus connects to i2c bus 2. 471 472The MIPI CSI-2 OV5640 module is connected to the i.MX internal MIPI CSI-2 473receiver, and the four virtual channel outputs from the receiver are 474routed as follows: vc0 to the IPU1 CSI0 mux, vc1 directly to IPU1 CSI1, 475vc2 directly to IPU2 CSI0, and vc3 to the IPU2 CSI1 mux. The OV5640 is 476also connected to i2c bus 2 on the SabreLite, therefore the OV5642 and 477OV5640 must not share the same i2c slave address. 478 479The following basic example configures unprocessed video capture 480pipelines for both sensors. The OV5642 is routed to ipu1_csi0, and 481the OV5640, transmitting on MIPI CSI-2 virtual channel 1 (which is 482imx6-mipi-csi2 pad 2), is routed to ipu1_csi1. Both sensors are 483configured to output 640x480, and the OV5642 outputs YUYV2X8, the 484OV5640 UYVY2X8: 485 486.. code-block:: none 487 488 # Setup links for OV5642 489 media-ctl -l "'ov5642 1-0042':0 -> 'ipu1_csi0_mux':1[1]" 490 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" 491 media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]" 492 # Setup links for OV5640 493 media-ctl -l "'ov5640 1-0040':0 -> 'imx6-mipi-csi2':0[1]" 494 media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]" 495 media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]" 496 # Configure pads for OV5642 pipeline 497 media-ctl -V "'ov5642 1-0042':0 [fmt:YUYV2X8/640x480 field:none]" 498 media-ctl -V "'ipu1_csi0_mux':2 [fmt:YUYV2X8/640x480 field:none]" 499 media-ctl -V "'ipu1_csi0':2 [fmt:AYUV32/640x480 field:none]" 500 # Configure pads for OV5640 pipeline 501 media-ctl -V "'ov5640 1-0040':0 [fmt:UYVY2X8/640x480 field:none]" 502 media-ctl -V "'imx6-mipi-csi2':2 [fmt:UYVY2X8/640x480 field:none]" 503 media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/640x480 field:none]" 504 505Streaming can then begin independently on the capture device nodes 506"ipu1_csi0 capture" and "ipu1_csi1 capture". The v4l2-ctl tool can 507be used to select any supported YUV pixelformat on the capture device 508nodes, including planar. 509 510SabreAuto with ADV7180 decoder 511------------------------------ 512 513On the SabreAuto, an on-board ADV7180 SD decoder is connected to the 514parallel bus input on the internal video mux to IPU1 CSI0. 515 516The following example configures a pipeline to capture from the ADV7180 517video decoder, assuming NTSC 720x480 input signals, with Motion 518Compensated de-interlacing. Pad field types assume the adv7180 outputs 519"interlaced". $outputfmt can be any format supported by the ipu1_ic_prpvf 520entity at its output pad: 521 522.. code-block:: none 523 524 # Setup links 525 media-ctl -l "'adv7180 3-0021':0 -> 'ipu1_csi0_mux':1[1]" 526 media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]" 527 media-ctl -l "'ipu1_csi0':1 -> 'ipu1_vdic':0[1]" 528 media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]" 529 media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]" 530 media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]" 531 # Configure pads 532 media-ctl -V "'adv7180 3-0021':0 [fmt:UYVY2X8/720x480]" 533 media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/720x480 field:interlaced]" 534 media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/720x480 field:interlaced]" 535 media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/720x480 field:none]" 536 media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/720x480 field:none]" 537 media-ctl -V "'ipu1_ic_prpvf':1 [fmt:$outputfmt field:none]" 538 539Streaming can then begin on the capture device node at 540"ipu1_ic_prpvf capture". The v4l2-ctl tool can be used to select any 541supported YUV or RGB pixelformat on the capture device node. 542 543This platform accepts Composite Video analog inputs to the ADV7180 on 544Ain1 (connector J42). 545 546SabreSD with MIPI CSI-2 OV5640 547------------------------------ 548 549Similarly to SabreLite, the SabreSD supports a parallel interface 550OV5642 module on IPU1 CSI0, and a MIPI CSI-2 OV5640 module. The OV5642 551connects to i2c bus 1 and the OV5640 to i2c bus 2. 552 553The device tree for SabreSD includes OF graphs for both the parallel 554OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI 555CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled. 556The OV5640 module connects to MIPI connector J5 (sorry I don't have the 557compatible module part number or URL). 558 559The following example configures a direct conversion pipeline to capture 560from the OV5640, transmitting on MIPI CSI-2 virtual channel 1. $sensorfmt 561can be any format supported by the OV5640. $sensordim is the frame 562dimension part of $sensorfmt (minus the mbus pixel code). $outputfmt can 563be any format supported by the ipu1_ic_prpenc entity at its output pad: 564 565.. code-block:: none 566 567 # Setup links 568 media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]" 569 media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]" 570 media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]" 571 media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]" 572 media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]" 573 # Configure pads 574 media-ctl -V "'ov5640 1-003c':0 [fmt:$sensorfmt field:none]" 575 media-ctl -V "'imx6-mipi-csi2':2 [fmt:$sensorfmt field:none]" 576 media-ctl -V "'ipu1_csi1':1 [fmt:AYUV32/$sensordim field:none]" 577 media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/$sensordim field:none]" 578 media-ctl -V "'ipu1_ic_prpenc':1 [fmt:$outputfmt field:none]" 579 580Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl 581tool can be used to select any supported YUV or RGB pixelformat on the 582capture device node. 583 584 585Known Issues 586------------ 587 5881. When using 90 or 270 degree rotation control at capture resolutions 589 near the IC resizer limit of 1024x1024, and combined with planar 590 pixel formats (YUV420, YUV422p), frame capture will often fail with 591 no end-of-frame interrupts from the IDMAC channel. To work around 592 this, use lower resolution and/or packed formats (YUYV, RGB3, etc.) 593 when 90 or 270 rotations are needed. 594 595 596File list 597--------- 598 599drivers/staging/media/imx/ 600include/media/imx.h 601include/linux/imx-media.h 602 603References 604---------- 605 606.. [#f1] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6DQRM.pdf 607.. [#f2] http://www.nxp.com/assets/documents/data/en/reference-manuals/IMX6SDLRM.pdf 608 609 610Authors 611------- 612 613- Steve Longerbeam <steve_longerbeam@mentor.com> 614- Philipp Zabel <kernel@pengutronix.de> 615- Russell King <linux@armlinux.org.uk> 616 617Copyright (C) 2012-2017 Mentor Graphics Inc. 618