# V4L2 Video Encoder

NVIDIA Tegra V4L2 Video Encoder Description and Extensions. More...

## Classes

struct | v4l2_enc_slice_length_param_
| Holds encoder slice length parameters. More...
struct | v4l2_enc_virtual_buffer_size_
| Holds encoder virtual buffer size parameters. More...
struct | v4l2_enc_num_ref_frames_
| Holds encoder number of reference frame parameters. More...
struct | v4l2_enc_slice_intrarefresh_param_
| Holds encoder slice intrareferesh parameters. More...
struct | v4l2_enc_ROI_param_
| Holds the encoder quality parameters for a single ROI region. More...
struct | v4l2_enc_frame_ROI_params_
| Holds the encoder frame ROI parameters. More...
struct | MVInfo_
| Holds the motion vector parameters for a single macro block. More...
struct | v4l2_ctrl_videoenc_outputbuf_metadata_MV_
| Holds the motion vector parameters for one complete frame. More...
struct | v4l2_ctrl_videoenc_outputbuf_metadata_
| Holds the encoder output metadata for a frame. More...
struct | _v4l2_ctrl_video_metadata
| Holds the metadata parameters for video encoder and decoder. More...
struct | _v4l2_ctrl_video_qp_range
| Holds the encoder frame min/max QP parameters. More...

## Defines

#define | V4L2_CID_MPEG_VIDEOENC_TEMPORAL_TRADEOFF_LEVEL (V4L2_CID_MPEG_BASE+525)
| Control ID to configure encoder to drop frames while encoding.
#define | V4L2_CID_MPEG_VIDEOENC_SLICE_LENGTH_PARAM (V4L2_CID_MPEG_BASE+526)
| Control ID to configure encoder slice length either in terms of MBs or bits.
#define | V4L2_CID_MPEG_VIDEOENC_ROI_PARAMS (V4L2_CID_MPEG_BASE+527)
| Control ID to configure encoder to encode particular region of frame in high quality.
#define | V4L2_CID_MPEG_VIDEOENC_VIRTUALBUFFER_SIZE (V4L2_CID_MPEG_BASE+528)
| Control ID to specify virtual buffer size in bits for encoder.
#define | V4L2_CID_MPEG_VIDEOENC_NUM_REFERENCE_FRAMES (V4L2_CID_MPEG_BASE+529)
| Control ID to specify maximum number of reference frames that can be used.
#define | V4L2_CID_MPEG_VIDEOENC_SLICE_INTRAREFRESH_PARAM (V4L2_CID_MPEG_BASE+530)
| Control ID to specify the encoder slice intra refresh interval.
#define | V4L2_CID_MPEG_VIDEOENC_NUM_BFRAMES (V4L2_CID_MPEG_BASE+531)
| Control ID to set number of B frames to be encoded between two P frames.
#define | V4L2_CID_MPEG_VIDEOENC_INSERT_SPS_PPS_AT_IDR (V4L2_CID_MPEG_BASE+532)
| Control ID to enable/disable inserting SPS and PPS explicitly at IDR interval.
#define | V4L2_CID_MPEG_VIDEOENC_METADATA (V4L2_CID_MPEG_BASE+533)
| Control ID to get encoder output metadata.
#define | V4L2_CID_MPEG_VIDEOENC_ENABLE_METADATA_MV (V4L2_CID_MPEG_BASE+534)
| Control ID to enable/disable encoder Motion Vector reporting.
#define | V4L2_CID_MPEG_VIDEOENC_METADATA_MV (V4L2_CID_MPEG_BASE+535)
| Control ID to get encoder output motion vector metadata.
#define | V4L2_CID_MPEG_VIDEOENC_QP_RANGE (V4L2_CID_MPEG_BASE+536)
| Control ID to set QP range for I/P/B frames.
#define | V4L2_MAX_ROI_REGIONS 8
| Maximum number of ROI regions supported by the encoder.
#define | QP_RETAIN_VAL -1
| Setting Qp values in v4l2_ctrl_video_qp_range to QP_RETAIN_VAL will retain default or previously set QP values.

## Typedefs

typedef struct v4l2_enc_slice_length_param_ | v4l2_enc_slice_length_param
| Holds encoder slice length parameters.
typedef struct v4l2_enc_virtual_buffer_size_ | v4l2_enc_virtual_buffer_size
| Holds encoder virtual buffer size parameters.
typedef struct v4l2_enc_num_ref_frames_ | v4l2_enc_num_ref_frames
| Holds encoder number of reference frame parameters.
typedef struct v4l2_enc_slice_intrarefresh_param_ | v4l2_enc_slice_intrarefresh_param
| Holds encoder slice intrareferesh parameters.
typedef struct v4l2_enc_ROI_param_ | v4l2_enc_ROI_param
| Holds the encoder quality parameters for a single ROI region.
typedef struct v4l2_enc_frame_ROI_params_ | v4l2_enc_frame_ROI_params
| Holds the encoder frame ROI parameters.
typedef struct MVInfo_ | MVInfo
| Holds the motion vector parameters for a single macro block.
typedef struct v4l2_ctrl_videoenc_outputbuf_metadata_MV_ | v4l2_ctrl_videoenc_outputbuf_metadata_MV
| Holds the motion vector parameters for one complete frame.
typedef struct v4l2_ctrl_videoenc_outputbuf_metadata_ | v4l2_ctrl_videoenc_outputbuf_metadata
| Holds the encoder output metadata for a frame.
typedef struct _v4l2_ctrl_video_metadata | v4l2_ctrl_video_metadata
| Holds the metadata parameters for video encoder and decoder.
typedef struct _v4l2_ctrl_video_qp_range | v4l2_ctrl_video_qp_range
| Holds the encoder frame min/max QP parameters.

## Enumerations

enum | v4l2_enc_temporal_tradeoff_level_type { V4L2_ENC_TEMPORAL_TRADEOFF_LEVEL_DROPNONE = 0, V4L2_ENC_TEMPORAL_TRADEOFF_LEVEL_DROP1IN5, V4L2_ENC_TEMPORAL_TRADEOFF_LEVEL_DROP1IN3, V4L2_ENC_TEMPORAL_TRADEOFF_LEVEL_DROP1IN2, V4L2_ENC_TEMPORAL_TRADEOFF_LEVEL_DROP2IN3 }
| Enum specifying the types of encoder temporal tradeoff levels. More...
enum | v4l2_enc_slice_length_type { V4L2_ENC_SLICE_LENGTH_TYPE_BITS = 0, V4L2_ENC_SLICE_LENGTH_TYPE_MBLK }
| Enum specifying the type of slice length. More...

## Detailed Description

NVIDIA Tegra V4L2 Video Encoder Description and Extensions.

The video encoder device node is "/dev/nvhost-msenc".

### Supported pixelformats:

OUTPUT PLANE | CAPTURE PLANE :---------------------: | :-------------- V4L2_PIX_FMT_YUV420M | V4L2_PIX_FMT_H264

- | V4L2_PIX_FMT_H265

### Supported memory types:

MEMORY | OUTPUT PLANE | CAPTURE PLANE :------------------: | :----------: | :-----------: V4L2_MEMORY_MMAP | Y | Y V4L2_MEMORY_DMABUF | Y | N V4L2_MEMORY_USERPTR | N | N

Attention:
  For the video encoder, it is necessary that the capture plane format be set before the output plane format and only then request buffers on any of the planes.

### Supported controls:

#### Controls from the open souce v4l2-controls header: Control Id | Purpose | Runtime Configurable -------------------------------- | -------------------- | :------------------: V4L2_CID_MPEG_VIDEO_BITRATE | Bitrate | Y V4L2_CID_MPEG_VIDEO_H264_PROFILE | H.264 Encode Profile | N V4L2_CID_MPEG_VIDEO_BITRATE_MODE | Rate Control Mode | N V4L2_CID_MPEG_VIDEO_GOP_SIZE | I-frame Interval | N V4L2_CID_MPEG_VIDEO_H264_LEVEL | Encode Level | N V4L2_CID_MPEG_MFC51_VIDEO_FORCE_FRAME_TYPE | Force I-frame on one of queued output plane buffer | Y

All non-runtime configurable options should be set after setting formats on both the planes and before requesting buffers on either plane.

The runtime configurable parameters can be called anytime after setting formats on both the planes.

#### NVidia specific controls:

- V4L2_CID_MPEG_VIDEO_H265_PROFILE
- V4L2_CID_MPEG_VIDEO_IDR_INTERVAL
- V4L2_CID_MPEG_VIDEOENC_TEMPORAL_TRADEOFF_LEVEL
- V4L2_CID_MPEG_VIDEOENC_SLICE_LENGTH_PARAM
- V4L2_CID_MPEG_VIDEOENC_ROI_PARAMS
- V4L2_CID_MPEG_VIDEOENC_VIRTUALBUFFER_SIZE
- V4L2_CID_MPEG_VIDEOENC_NUM_REFERENCE_FRAMES
- V4L2_CID_MPEG_VIDEOENC_SLICE_INTRAREFRESH_PARAM
- V4L2_CID_MPEG_VIDEOENC_NUM_BFRAMES
- V4L2_CID_MPEG_VIDEOENC_INSERT_SPS_PPS_AT_IDR
- V4L2_CID_MPEG_VIDEOENC_METADATA
- V4L2_CID_MPEG_VIDEOENC_METADATA_MV
- V4L2_CID_MPEG_VIDEOENC_ENABLE_METADATA_MV
- V4L2_CID_MPEG_VIDEOENC_QP_RANGE

#### Setting Framerate: Encoder framerate can be set with VIDIOC_S_PARM ioctl by setting the numerator and denominator in `v4l2_streamparm.parm.output.timeperframe`.

### Supported Encoder Profiles: #### H.264

- V4L2_MPEG_VIDEO_H264_PROFILE_MAIN
- V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE
- V4L2_MPEG_VIDEO_H264_PROFILE_HIGH

#### H.265

- V4L2_MPEG_VIDEO_H265_PROFILE_MAIN

### Encoder output metadata: Encoder supports reporting frame related metadata including motion vectors for that frame. See V4L2_CID_MPEG_VIDEOENC_METADATA, V4L2_CID_MPEG_VIDEOENC_METADATA_MV and V4L2_CID_MPEG_VIDEOENC_ENABLE_METADATA_MV for more information.

### EOS Handling: Following sequence should be followed for sending EOS and recieving EOS from the encoder.

1. Send EOS to encoder by queueing on the output plane a buffer with bytesused = 0 for the 0th plane (`v4l2_buffer.m.planes[0].bytesused = 0`).
2. Dequeue buffers on the capture plane, till it gets a buffer with bytesused = 0 for the 0th plane.

## Define Documentation

#define QP_RETAIN_VAL -1

Setting Qp values in v4l2_ctrl_video_qp_range to QP_RETAIN_VAL will retain default or previously set QP values.

Definition at line 1138 of file v4l2_nv_extensions.h.

#define V4L2_CID_MPEG_VIDEOENC_ENABLE_METADATA_MV (V4L2_CID_MPEG_BASE+534)

Control ID to enable/disable encoder Motion Vector reporting.

A boolean value should be supplied with this control.

Attention:
  This control should be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 779 of file v4l2_nv_extensions.h.

#define V4L2_CID_MPEG_VIDEOENC_INSERT_SPS_PPS_AT_IDR (V4L2_CID_MPEG_BASE+532)

Control ID to enable/disable inserting SPS and PPS explicitly at IDR interval.

A boolean value should be supplied with this control.

Attention:
  This control should be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 757 of file v4l2_nv_extensions.h.

#define V4L2_CID_MPEG_VIDEOENC_METADATA (V4L2_CID_MPEG_BASE+533)

Control ID to get encoder output metadata.

A pointer to valid v4l2_ctrl_video_metadata structure should be supplied with this control.

Attention:
  This control should be read after dequeueing a buffer successfully from the capture plane. The values in the structure are valid till the buffer is queued again.

Definition at line 769 of file v4l2_nv_extensions.h.

#define V4L2_CID_MPEG_VIDEOENC_METADATA_MV (V4L2_CID_MPEG_BASE+535)

Control ID to get encoder output motion vector metadata.

A pointer to valid v4l2_ctrl_videoenc_outputbuf_metadata_MV structure should be supplied with this control.

Attention:
  This control should be read after dequeueing a buffer successfully from the capture plane. The values in the structure are valid till the buffer is queued again.

Definition at line 791 of file v4l2_nv_extensions.h.

#define V4L2_CID_MPEG_VIDEOENC_NUM_BFRAMES (V4L2_CID_MPEG_BASE+531)

Control ID to set number of B frames to be encoded between two P frames.

Works only with H.264 encoder. An integer value should be supplied with this control.

Attention:
  This control should be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 747 of file v4l2_nv_extensions.h.

#define V4L2_CID_MPEG_VIDEOENC_NUM_REFERENCE_FRAMES (V4L2_CID_MPEG_BASE+529)

Control ID to specify maximum number of reference frames that can be used.

An integer value should be supplied with this control.

Attention:
  This control should be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 725 of file v4l2_nv_extensions.h.

#define V4L2_CID_MPEG_VIDEOENC_QP_RANGE (V4L2_CID_MPEG_BASE+536)

Control ID to set QP range for I/P/B frames.

A pointer to valid v4l2_ctrl_video_qp_range structure should be supplied with this control.

Attention:
  This control should be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 802 of file v4l2_nv_extensions.h.

#define V4L2_CID_MPEG_VIDEOENC_ROI_PARAMS (V4L2_CID_MPEG_BASE+527)

Control ID to configure encoder to encode particular region of frame in high quality.

A pointer to a valid v4l2_enc_frame_ROI_params structure should be supplied with this control.

Attention:
  This control should be set after requesting buffers on both the planes.

Definition at line 704 of file v4l2_nv_extensions.h.

#define V4L2_CID_MPEG_VIDEOENC_SLICE_INTRAREFRESH_PARAM (V4L2_CID_MPEG_BASE+530)

Control ID to specify the encoder slice intra refresh interval.

A pointer to a valid v4l2_enc_slice_intrarefresh_param structure should be supplied with this control.

Attention:
  This control should be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 736 of file v4l2_nv_extensions.h.

#define V4L2_CID_MPEG_VIDEOENC_SLICE_LENGTH_PARAM (V4L2_CID_MPEG_BASE+526)

Control ID to configure encoder slice length either in terms of MBs or bits.

A pointer to a valid v4l2_enc_slice_length_param structure should be supplied with this control.

Attention:
  This control should be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 692 of file v4l2_nv_extensions.h.

#define V4L2_CID_MPEG_VIDEOENC_TEMPORAL_TRADEOFF_LEVEL (V4L2_CID_MPEG_BASE+525)

Control ID to configure encoder to drop frames while encoding.

A value of type v4l2_enc_temporal_tradeoff_level_type should be supplied with this control.

Attention:
  This control should be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 681 of file v4l2_nv_extensions.h.

#define V4L2_CID_MPEG_VIDEOENC_VIRTUALBUFFER_SIZE (V4L2_CID_MPEG_BASE+528)

Control ID to specify virtual buffer size in bits for encoder.

A pointer to a valid v4l2_enc_virtual_buffer_size structure should be supplied with this control.

Attention:
  This control should be set after setting formats on both the planes and before requesting buffers on either plane.

Definition at line 715 of file v4l2_nv_extensions.h.

#define V4L2_MAX_ROI_REGIONS 8

Maximum number of ROI regions supported by the encoder.

Definition at line 1030 of file v4l2_nv_extensions.h.

## Typedef Documentation

typedef struct MVInfo_ MVInfo

Holds the motion vector parameters for a single macro block.

typedef struct _v4l2_ctrl_video_metadata v4l2_ctrl_video_metadata

Holds the metadata parameters for video encoder and decoder.

The metadata is valid for the buffer with index buffer_index after the buffer is dequeued till it is queued again.

typedef struct _v4l2_ctrl_video_qp_range v4l2_ctrl_video_qp_range

Holds the encoder frame min/max QP parameters.

Should be used with V4L2_CID_MPEG_VIDEOENC_QP_RANGE ioctl.

typedef struct v4l2_ctrl_videoenc_outputbuf_metadata_ v4l2_ctrl_videoenc_outputbuf_metadata

Holds the encoder output metadata for a frame.

To be used with V4L2_CID_MPEG_VIDEOENC_METADATA ioctl.

typedef struct v4l2_ctrl_videoenc_outputbuf_metadata_MV_ v4l2_ctrl_videoenc_outputbuf_metadata_MV

Holds the motion vector parameters for one complete frame.

typedef struct v4l2_enc_frame_ROI_params_ v4l2_enc_frame_ROI_params

Holds the encoder frame ROI parameters.

Should be used with V4L2_CID_MPEG_VIDEOENC_ROI_PARAMS ioctl.

typedef struct v4l2_enc_num_ref_frames_ v4l2_enc_num_ref_frames

Holds encoder number of reference frame parameters.

To be used with V4L2_CID_MPEG_VIDEOENC_NUM_REFERENCE_FRAMES ioctl.

This is not supported for H265

typedef struct v4l2_enc_ROI_param_ v4l2_enc_ROI_param

Holds the encoder quality parameters for a single ROI region.

typedef struct v4l2_enc_slice_intrarefresh_param_ v4l2_enc_slice_intrarefresh_param

Holds encoder slice intrareferesh parameters.

To be used with V4L2_CID_MPEG_VIDEOENC_SLICE_INTRAREFRESH_PARAM ioctl.

typedef struct v4l2_enc_slice_length_param_ v4l2_enc_slice_length_param

Holds encoder slice length parameters.

To be used with V4L2_CID_MPEG_VIDEOENC_SLICE_LENGTH_PARAM ioctl.

typedef struct v4l2_enc_virtual_buffer_size_ v4l2_enc_virtual_buffer_size

Holds encoder virtual buffer size parameters.

To be used with V4L2_CID_MPEG_VIDEOENC_VIRTUALBUFFER_SIZE ioctl.

## Enumeration Type Documentation

enum v4l2_enc_slice_length_type

Enum specifying the type of slice length.

Enumerator:

  V4L2_ENC_SLICE_LENGTH_TYPE_BITS | Slice size is specified in terms of number of bytes.
  V4L2_ENC_SLICE_LENGTH_TYPE_MBLK | Slice size is specified in terms of number of macroblocks.

Definition at line 972 of file v4l2_nv_extensions.h.

enum v4l2_enc_temporal_tradeoff_level_type

Enum specifying the types of encoder temporal tradeoff levels.

Enumerator:

  V4L2_ENC_TEMPORAL_TRADEOFF_LEVEL_DROPNONE | Do not drop any buffers.
  V4L2_ENC_TEMPORAL_TRADEOFF_LEVEL_DROP1IN5 | Drop 1 in every 5 buffers.
  V4L2_ENC_TEMPORAL_TRADEOFF_LEVEL_DROP1IN3 | Drop 1 in every 3 buffers.
  V4L2_ENC_TEMPORAL_TRADEOFF_LEVEL_DROP1IN2 | Drop 1 in every 2 buffers.
  V4L2_ENC_TEMPORAL_TRADEOFF_LEVEL_DROP2IN3 | Drop 2 in every 3 buffers.

Definition at line 956 of file v4l2_nv_extensions.h.