L4T Multimedia API Reference

27.1 Release

 All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
Camera Capture with TensorRT and Multi-Channel Encoding

Requirements

  • Camera
  • Display

Overview

This sample demonstrates four different resolutions of video capture coming out of camera sensor directly. Among these streams, three of them go through the encoding process and are saved as H.264 or H.265 file. The fourth stream provides full resolution and uses NVIDIA® TensorRT to perform object detection. If target object is detected, it will draw a bounding box around the object on the frame. Finally, the frame with the bounding box is rendered on the display and simultaneously encoded as an H.264 or H.265 file.

Detailed Buffer Management

Detailed Buffer Management

Buffer Flow of the Frontend Using GIE

Buffer Flow of the Frontend Using GIE

Key Classes and Methods

For information on libargus, see Libargus API. For information on EGLStream, see documentation on the Khronos web site.

StreamConsumer Class

ClassDescription
StreamConsumerA base class of the libargus OutputStream consumer. It contains a dedicated thread to poll frames from Argus::OutputStream.

The following table describes key StreamConsumer members.

MembersDescription
setOutputStreamSets OutputStream which be used to create the FrameConsumer.
getSizeGets stream resolution.
threadInitializeVirtual method that initializes the consumer thread.
threadExecuteVirtual method that manages the thread loop.
threadShutdownVirtual method that destroys the consumer thread.
processFrameVirtual method that handles one frame.

VideoEncoder Class

ClassDescription
VideoEncoderThis is a utility class to simplify the use of V4L2 video encoder.

The following table describes key VideoEncoder members.

MembersDescription
VideoEncoderConstructor in which to specify resolution, output video filename, and select encoding format.
initializeInitialize video encoder and stream consumer.
shutdownDestroy resources.
setBufferDoneCallbackSet callback to return buffer to caller.
encodeFromFdFeed the encoder with a new buffer.

VideoStreamConsumer Class

ClassDescription
VideoStreamConsumerThis class is a derived class of StreamConsumer to support video encoding. It reuses the VideoEncoder object for video encoding.

The following table describes key VideoStreamConsumer members.

MembersDescription
VideoEncodeStreamConsumerConstructor in which to specify the name of the consumer, output video filename and select encoding format.
threadInitializeInitializes the video encoder and stream consumer.
threadShutdownDestroys resources.
processFrameReceives frames and pushes them to the encoder.

GIEStreamConsumer Class

ClassDescription
GIEStreamConsumerThis class is a derived class of StreamConsumer to support TensorRT (GIE) inference. It also has a VideoEncoder object to support video encoding.

The following table describes key GIEStreamConsumer members.

MembersDescription
GIEStreamConsumerConstructor in which to specify the name of the consumer, output video filename and select encoding format.
threadInitializeInitializes TensorRT (GIE) context, video encoder and stream consumer.
threadShutdownDestroys resources.
processFrameReceives frames and prepares them for the TensorRT (GIE) inference.
setDeployFileSets the deploy file for building the TensorRT (GIE) context.
setModelFileSets the model file for building the TensorRT (GIE) context.
RenderThreadProcLoops to draw the bounding box, render the frame, and encode the video.
GIEThreadProcLoops to do the TensorRT (GIE) inference.

Command Line Options

./frontend [OPTIONS]

Enter the following command to view the supported options.

./frontend -h

Commands

Press 'q' to exit.