L4T Multimedia API Reference

28.1 Release

 All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
multimedia_api/ll_samples/docs/l4t_mm_camcap_tensorrt_and_multi_enc.md
Go to the documentation of this file.
1 Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
2 <!-- Sample is in the frontend directory. -->
3 
4 @page l4t_mm_camcap_tensorrt_multichannel_group Frontend
5 @{
6 
7  - [Overview](#overview)
8  - [Building and Running](#build_and_run)
9  - [Flow](#flow)
10  - [Key Classes and Methods](#key)
11 
12 #### Requirements ####
13 - Camera
14 - Display
15 
16 <a name="overview">
17 - - - - - - - - - - - - - - -
18 ## Overview ##
19 
20 This sample demonstrates video capture for four video streams coming directly
21 from camera sensors, where the streams have different resolutions.
22 Three of the streams go through the
23 encoding process and are saved as H.264 or H.265 files. The fourth stream
24 provides full resolution and uses NVIDIA<sup>&reg;</sup> TensorRT<sup>&tm;</sup>
25 to perform object detection. If target object is detected, it will draw a
26 bounding box around the object on the frame. Finally, the frame with the
27 bounding box is rendered on the display and simultaneously encoded as an H.264
28 or H.265 file.
29 
30 <a name="build_and_run">
31 - - - - - - - - - - - - - - -
32 ## Building and Running ##
33 
34 ### Prerequisites ###
35 * You have followed Steps 1-3 in @ref mmapi_build.
36 
37 ### To build
38 1. Ensure the following variable is set to 1 by making the
39  following change in the Makefile:
40 
41  ENABLE_TRT := 1
42 2. Enter:
43 
44  $ cd frontend
45  $ make
46 
47 ### To run
48 * Enter:
49 
50  $ ./frontend --deploy ../../data/Model/GoogleNet_three_class/GoogleNet_modified_threeClass_VGA.prototxt \
51  --model ../../data/Model/GoogleNet_three_class/GoogleNet_modified_threeClass_VGA.caffemodel \
52  --no-preview
53 
54 ### To quit
55 * Enter `q`.
56 
57 ### To view command-line options:
58 * Enter:
59 
60  $ cd frontend
61  $ ./frontend -h
62 
63 
64 <a name="flow">
65 - - - - - - - - - - - - - - -
66 ## Flow
67 The following diagrams illustrate the flow of data through this sample.
68 
69 #### Detailed Buffer Management ####
70 ![Detailed Buffer Management](l4t_mm_camcap_tensorrt.png)
71 
72 #### Buffer Flow of the Frontend Using TensorRT ####
73 ![Buffer Flow of the Frontend Using TensorRT](l4t_mm_camcap_buff_flow_tensorrt.png)
74 
75  $ ./frontend --deploy ../../data/Model/GoogleNet_three_class/GoogleNet_modified_threeClass_VGA.prototxt \
76  --model ../../data/Model/GoogleNet_three_class/GoogleNet_modified_threeClass_VGA.caffemodel
77 
78 <a name="key">
79 - - - - - - - - - - - - - - -
80 ## Key Classes and Methods ##
81 
82 For information on libargus, see @ref LibargusAPI. For information on EGLStream, see documentation on the Khronos web site.
83 
84 ### StreamConsumer Class ###
85 
86 The sample defines `StreamConsumer` as an abstracted class. It is not imported from other libraries.
87 
88 |Class|Description|
89 |-------|-----------|
90 |StreamConsumer|A base class of the libargus `OutputStream` consumer. It contains a dedicated thread to poll frames from Argus::OutputStream.|
91 
92 The following table describes key `StreamConsumer` members.
93 
94 |Members|Description|
95 |-------|-----------|
96 |setOutputStream|Sets OutputStream which be used to create the FrameConsumer.|
97 |getSize|Gets stream resolution.|
98 |threadInitialize|Virtual method that initializes the consumer thread.|
99 |threadExecute|Virtual method that manages the thread loop.|
100 |threadShutdown|Virtual method that destroys the consumer thread.|
101 |processFrame|Virtual method that handles one frame.|
102 
103 ### VideoEncoder Class ###
104 
105 The sample defines `VideoEncoder` as an abstracted class. It is not imported from other libraries.
106 
107 |Class|Description|
108 |-------|-----------|
109 |VideoEncoder|This is a utility class to simplify the use of V4L2 video encoder.|
110 
111 The following table describes key `VideoEncoder` members.
112 
113 |Members|Description|
114 |-------|-----------|
115 |VideoEncoder|Constructor in which to specify resolution, output video filename, and select encoding format.|
116 |initialize|Initialize video encoder and stream consumer.|
117 |shutdown|Destroy resources.|
118 |setBufferDoneCallback|Set callback to return buffer to caller.|
119 |encodeFromFd|Feed the encoder with a new buffer.|
120 
121 ### VideoStreamConsumer Class ###
122 
123 The sample defines `VideoStreamConsumer` as an abstracted class. It is not imported from other libraries.
124 
125 |Class|Description|
126 |-------|-----------|
127 |VideoStreamConsumer|This class is a derived class of StreamConsumer to support video encoding. It reuses the VideoEncoder object for video encoding.|
128 
129 The following table describes key `VideoStreamConsumer` members.
130 
131 |Members|Description|
132 |-------|-----------|
133 |VideoEncodeStreamConsumer|Constructor in which to specify the name of the consumer, output video filename and select encoding format.|
134 |threadInitialize|Initializes the video encoder and stream consumer.|
135 |threadShutdown|Destroys resources.|
136 |processFrame|Receives frames and pushes them to the encoder.|
137 
138 ### TRTStreamConsumer Class ###
139 
140 The sample defines `TRTStreamConsumer` as an abstracted class. It is not imported from other libraries.
141 
142 
143 |Class|Description|
144 |-------|-----------|
145 |TRTStreamConsumer|This class is a derived class of StreamConsumer to support TensorRT (GIE) inference. It also has a VideoEncoder object to support video encoding.|
146 
147 The following table describes key `TRTStreamConsumer` members.
148 
149 |Members|Description|
150 |-------|-----------|
151 |TRTStreamConsumer|Constructor in which to specify the name of the consumer, output video filename and select encoding format.|
152 |threadInitialize|Initializes TensorRT context, video encoder and stream consumer.|
153 |threadShutdown|Destroys resources.|
154 |processFrame|Receives frames and prepares them for the TensorRT inference.|
155 |setDeployFile|Sets the deploy file for building the TensorRT context.|
156 |setModelFile|Sets the model file for building the TensorRT context.|
157 |RenderThreadProc|Loops to draw the bounding box, render the frame, and encode the video.|
158 |TRTThreadProc|Loops to do the TensorRT inference.|
159