L4T Multimedia API Reference

27.1 Release

 All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
nvee_video_dec_tensorrt.md
Go to the documentation of this file.
1 Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
2 
3 @page ee_vid_decode_gie TensorRT Sample with Arbitrary Batch Size
4 @{
5 
6  - [Overview](#overview)
7  - [Key Structure and Classes](#key)
8  - [Key Thread](#key_thread)
9  - [Programming Notes](#notes)
10  - [Command Line Options](#options)
11 
12 - - - - - - - - - - - - - - -
13 <a name="overview">
14 ## Overview ##
15 
16 This sample demonstrates the simplest way to use NVIDIA<sup>&reg;</sup>
17 TensorRT<sup>&tm;</sup> to decode video and save the bounding box information to
18 the `result.txt` file. TensorRT was previously known as GPU Inference Engine
19 (GIE).
20 
21 The data pipeline is as follow:
22 
23  Input video file -> Decoder -> VIC -> TensorRT Inference -> Plain text file with Bounding Box info
24 
25 #### Operation Flow ####
26 
27 The sample does the following:
28 
29 1. Encodes the input video stream.
30 2. Performs one-channel video decodeVIC, which does the following:
31  - Converts the buffer layout from block linear to pitch linear.
32  - Scales the image resolution to what's required by TensorRT.
33 3. Uses TensorRT performs object identification and adds a bounding box to identified
34  object of original frame.
35 4. Converts YUV to RGB format and save it in a file.
36 
37 ![](nvee_video_dec_tensorrt.jpg)
38 
39 The block diagram contains not only the pipeline, but also the memory sharing information among different engines, which can be a reference for the other samples.
40 
41 #### Additional Details
42 
43 - README provides more details about how to build, run and implement this sample.
44 
45 #### Prerequisites ####
46 
47 Before running the sample, you must have the following:
48 
49 - CUDA
50 - TensorRT (previously known as GPU Inference Engine (GIE))
51 - OpenCV4Tegra
52 
53 This samples does not require a Camera or display.
54 
55 - - - - - - - - - - - - - - -
56 <a name="key">
57 ## Key Structure and Classes ##
58 
59 This sample uses the following key structures and classes:
60 
61 The global structure `context_t` manages all the resources in the application.
62 
63 |Element|Description|
64 |---|---|
65 |NvVideoDecoder|Contains all video decoding-related elements and functions.|
66 |NvVideoConverter|Contains elements and functions for video format conversion.|
67 |EGLDisplay|The EGLImage used for CUDA processing.|
68 |conv_output_plane_buf_queue|Output plane queue for video conversion.|
69 |GIE_Context|Provide a series of interfaces to load Caffemodel and do inference.|
70 
71 
72 <a name="key_thread">
73 ## Key Thread ##
74 
75 |Member|Description|
76 |---|---|
77 |decCaptureLoop|Get buffers from dec capture plane and push to converter, and handle resolution change.|
78 |Conv outputPlane dqThread|Return the buffers dequeued from converter output plane to decoder capture plane.|
79 |Conv captuerPlane dqThread|Get buffers from conv capture plane and push to the TensorRT buffer queue.|
80 |gieThread|CUDA process and doing inference.|
81 
82 <a name="notes">
83 ## Programming Notes ##
84 
85 
86 - The result saves the normalized rectangle within [0,1].
87 - Can use 02_video_dec_cuda sample to display and verify the result and scale the rectangle parameters with the following command:
88 
89  $ ./video_dec_cuda <in-file> <in-format> --bbox-file result.txt
90 
91 - Supports in-stream resolution changes.
92 - The default deploy file is
93 
94  GoogleNet-modified.prototxt
95 
96  The default model file is
97 
98  GoogleNet-modified-online_iter_30000.caffemodel
99 
100  In this directory:
101 
102  $SDKDIR/data/model
103 
104 - The batch size can be changed in GoogleNet-modified.prototxt.
105  The batch size is limited by the memory.
106  The biggest batch size for Ivanet is less than 40.
107 - End-of-stream (EOS) process:
108 
109  a. Completely read the file.
110 
111  b. Push a null `v4l2buf` to decoder.
112 
113  c. Decoder waits for all output plane buffers to return.
114 
115  d. Set `get_eos`:
116 
117  decCap thread exit
118 
119  e. End the TensorRT thread.
120 
121  f. Send EOS to the converter:
122 
123  conv output plane dqThread callback return false
124  conv output plane dqThread exit
125  conv capture plane dqThread callback return false
126  conv capture plane dqThread exit
127 
128  g. Delete the decoder:
129 
130  deinit output plane and capture plane buffers
131 
132  h. Delete the converter:
133 
134  unmap capture plane buffers
135 
136 
137 <a name="options">
138 ## Command Line Options ##
139 
140  ./video_dec_gie <in-file> <in-format> [options]
141 
142 Use the `-h` option to view the currently supported options.
143 
144 @}
145 
146 
147