L4T Multimedia API Reference

27.1 Release

 All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
CAFFE to TensorRT Model Sample

Overview

TensorRT is an accelerated neural network inference engine and run-time library. ConvertCaffeToGieModel is a standalone model conversion tool that converts a CAFFE network to a TensorRT compatible model. This tool aims to run offline on the NVIDIA Jetson platform and provides a cached TensorRT model stream to prevent subsequent repetitive network conversion. Using this converted model, TensorRT-based applications can improve greatly in accuracy and performance.

If the source model changes (i.e., is retrained), the tool performs conversion again to enable TensorRT accelerated inference.

Prerequisites

Before running the sample, you must have the following:

  • TensorRT package
  • A trained network

Key Structure and Classes

The CudaEngine structure is a TensorRT interface that invokes the TensorRT function.

The sample uses the following function:

Function Description
caffeToGIEModel Uses TensorRT API to transfer a network model from CAFFE to TensorRT.

Command Line Options

To run the sample, execute:

ConvertCaffeToGieModel -n <network_name> -l <model_name> -m <mode_name> -o <outBlob_name> [<outBlob_name>] -f <fp16 or fp32> -b <batch_size> -s <store_cache_name>

Use the -h option to view the currently supported options.

Example

The following example shows a sample execution:

./ConvertCaffeToGieModel -n ../../data/model/GoogleNet-modified.prototxt \
-l ../../data/model/GoogleNet-modified-online_iter_30000.caffemodel \
-m detection -o coverage,bboxes -f fp16 -b 2 \