stable/nvmedia_camera/nvmedia_camera_user_guide.md

Go to the documentation of this file.

<!-- Copyright (c) 2015-2016, NVIDIA CORPORATION.  All rights reserved. -->

@defgroup nvx_sample_nvmedia_camera (Vibrante) NvMedia Camera Capture Sample App
@brief NvMedia Camera Capture Sample user guide.
@ingroup nvx_samples
@{

<a name="Introduction">
## Introduction ##

@note This sample is available for Vibrante platform only.

`nvx_sample_nvmedia_camera` demonstrates frame grabbing from 1, 2, or 4 CSI cameras
with NvMedia API and basic VisionWorks-NvMedia interop facilities. As a basic
VisionWorks processing step, histogram equalization is used. Histogram equalization
is applied to the inner sub-rectangle only to make the effect of equalization more
obvious. The sample is tested with four OV10635 and OV10640 CSI cameras.

After VisionWorks installation, `nvx_sample_nvmedia_camera` is installed in the
following directory:

    /usr/share/visionworks/sources/samples/nvmedia_camera

For the steps to build sample applications, see the
@ref nvx_sample_building_linux section.

The sample organization on the global level is illustrated by the flowchart:

  - For Omni Vision 10635 cameras:


         [Capture2D step: output is YUV422]
                         |
         [Convertion YUV422 to RGBA step]
                         |
                    [EGLStream]
                         |
             [VisionWorks RGBX to NV12]
                         |
    [VisionWorks Histogram Equalization of Y plane]
                         |
             [VisionWorks NV12 to RGBX]
                         |
                 [NVXIO Rendering]


  - For Omni Vision 10640 cameras:


         [Capture2D step: output is RAW12]
                         |
    [NvMedia IPP post-processing: output is NV12]
                         |
                    [EGLStream]
                         |
             [VisionWorks RGBX to NV12]
                         |
    [VisionWorks Histogram Equalization of Y plane]
                         |
             [VisionWorks NV12 to RGBX]
                         |
                 [NVXIO Rendering]


The sample builds the chain of hardware-based NvMedia image processing steps that
are executed in a dedicated thread for each active camera. Thread communication
is based on queues without actual data copying. Communication between the NvMedia
step and VisionWorks is implemented with the EGLStream EGL extension. The main thread
implements histogram equalization and then rendering to exclude EGL and OpenVX context
sharing between threads.

## Attaching the Cameras ##

Cameras must be attached to the correct port to work. The following images
illustrate the camera connections.


@note There are multiple CSI connectors on the board. You must connect cameras
in the correct order:
    - For Jetson Pro, connect in ascending order starting from connector 0.
    - For NVIDIA<sup>&reg;</sup> DRIVE&tm; PX, attach the connectors in reverse order as shown in the
      DRIVE PX picture below.

@note For more details, see the *Platform Quick Start* for your board.

### DRIVE PX Camera Connections

DRIVE PX provides multiple video and camera ports. Before using these
cameras with the NvMedia sample applications, you must attach the cameras to
the ports in a specific order. If you fail to do so, NvMedia raises errors.

@warning Before connecting/disconnecting cameras to/from the platform, remove
power. Failure to do so may damage the platform.

![](docs-user/images/emb_vib_drivepx_front.jpg)

#### DRIVE PX Camera Interfaces

DRIVE PX provides GMSL, USB, and Ethernet camera interfaces. The GMSL camera
interfaces:

- Provide 12 simultaneous GMSL camera inputs
- Route camera data to both Tegras
- Control any camera from either Tegra
- Synchronize cameras within each group

#### Mapping Connectors to Tegra Chips

Each GMSL camera group can be routed to one or both Tegra chips via a GMSL
Deserializer.

![](docs-user/images/emb_nvmedia_drivepx_cam_to_tegras.jpg)

#### Connecting Cameras to DRIVE PX

When you connect cameras to a specific camera group, you must attach the
cameras in sequential order. For example, within Group A:

- Attach the first camera to A0.
- If used, attach the second camera to A1.
- If used, attach the second camera to A2.
- If used, attach the second camera to A3.

If you fail to attach cameras in this order, NvMedia generates an error message.

![](docs-user/images/emb_nvmedia_drivepx_cam_connector_map.jpg)

#### Default Aggregator Link Mapping

The following diagram shows how the digitized camera pictures are aggregated
into a frame, where the image from the first camera appears first in the frame
and the image from the last camera appears last in the frame. The frame size
changes depending on how many cameras are connected.

All the sample applications that process images provide a aggregate `n` option
to specify the number of aggregated images in the frame. In this option, `n`
maps as follows:

- n=1 maps to link0
- n=2 maps to link0 and link1
- etc.
- n=4 maps to all links

The number of cameras connected must match the number specified in the aggregate
option.

![](docs-user/images/emb_nvmedia_drivepx_cam_agg_link_map.jpg)

### Jetson TK1 Pro Camera Connections

![](docs-user/images/jetson-tk1-pro.png)

<a name="Executing">
## Executing the NvMedia Camera Sample ##

- On Jetson TK1 Pro with 2 OV10635 cameras, execute:

        $ ./nvx_sample_nvmedia_camera -c dvp-ov10635-yuv422-ab -n 2

- On DRIVE PX B00 with 2 OV10635 cameras, execute:

        $ ./nvx_sample_nvmedia_camera -c dvp-ov10635-yuv422-ab-e2379 -n 2

- On DRIVE PX B00 with 4 OV10640 cameras, execute:

        $ ./nvx_sample_nvmedia_camera -c dvp-ov10640-raw12-ab-e2379 -n 4

### Command Line Options ###

This topic provides a list of supported options and the values they consume.

#### \-c, \--camera ####
- Parameter: [Camera hardware configuration name]
- Description: Specifies the configuration name that describes the current
  camera setup.

- Usage: \--camera="dvp-ov10635-yuv422-ab"

#### \-n, \--camera_number ####
- Parameter: [Number of connected cameras]
- Description: Specifies the number of connected cameras. 1, 2, or 4 cameras are
  supported.
- Usage: \--camera_number=4

#### \-h, \--help ####
- Description: Prints the help message.

### Operational Keys ###
- Use `Space` to toggle histogram equalization mode.
- Use `ESC` to close the sample.

@}