Differences Between OpenCV and VisionWorks Primitives

This section illustrates differences between OpenCV and VisionWorks primitives.

Arithmetic Operations

For arithmetic operations (add, subtract, multiply), OpenCV always uses SATURATE conversion policy while VisionWorks allows to choice between SATURATE and WRAP conversion policies.

For the multiply operation, OpenCV always uses ROUND_TO_NEAREST_EVEN rounding policy while VisionWorks allows to choice between ROUND_TO_NEAREST_EVEN and ROUND_TO_ZERO rounding policies.

AccumulateWeighted VisionWorks primitive rounds to zero while cv::addWeighted rounds to nearest.

Filtering Operations

VisionWorks image filters (Box3x3, Gaussian3x3, etc.) uses VX_BORDER_MODE_UNDEFINED by default, which means that pixels on the border might be calculated inaccurately. OpenCV always uses some border extrapolation method. Use VX_BORDER_MODE_CONSTANT to match cv::BORDER_CONSTANT and VX_BORDER_MODE_REPLICATE to match cv::BORDER_REPLICATE behavior. VisionWorks doesn't have analogues for cv::BORDER_REFLECT, cv::BORDER_REFLECT101 and cv::BORDER_WRAP.

Color Conversion

VisionWorks uses BT.709 color space by default while OpenCV uses BT.601. It leads to different results in RGB[X]->Gray conversion if BT.601 is not explicitly used for VisionWorks images.

cv::Mat cv_src = cv::imread(fileName, cv::IMREAD_COLOR);
cv::cvtColor(cv_src, cv_src, cv::COLOR_BGR2RGB);

// OpenCV

cv::Mat cv_dst_original;
cv::cvtColor(cv_src, cv_dst_original, cv::COLOR_RGB2GRAY);

// VisionWorks

cv::Mat cv_dst_nvx(cv_src.size(), CV_8UC1);

vx_image vx_src = nvx_cv::createVXImageFromCVMat(context, cv_src);
vx_image vx_dst = nvx_cv::createVXImageFromCVMat(context, cv_dst_nvx);

vx_enum color_space = VX_COLOR_SPACE_BT601_625;
vxSetImageAttribute(vx_src, VX_IMAGE_ATTRIBUTE_SPACE, &color_space, sizeof(color_space));
vxSetImageAttribute(vx_dst, VX_IMAGE_ATTRIBUTE_SPACE, &color_space, sizeof(color_space));

vxuColorConvert(context, vx_src, vx_dst);

vxReleaseImage(&vx_src);
vxReleaseImage(&vx_dst);

Custom Convolution

The OpenCV cv::filter2D function and the VisionWorks Convolve primitive use different formulas.

VisionWorks:

\begin{eqnarray} k_0 &=& \frac{m}{2} \\ l_0 &=& \frac{n}{2} \\ sum(x,y) &=& \sum_{0 \leq k < m,0 \leq l < n} input(x+k_0-k, y+l_0-l) C_{k,l} \end{eqnarray}

OpenCV:

\begin{eqnarray} k_0 &=& \frac{m}{2} \\ l_0 &=& \frac{n}{2} \\ sum(x,y) &=& \sum_{0 \leq k < m,0 \leq l < n} input(x-k_0+k, y-l_0+l) C_{k,l} \end{eqnarray}

In order to port OpenCV code that uses cv::filter2D, the kernel must be reversed and converted into 16S type:

cv::Mat cv_src = cv::imread(fileName, cv::IMREAD_GRAYSCALE);
cv::Mat cv_kernel = getFilterKernel();

// OpenCV code

cv::Mat cv_dst_original(cv_src.size(), cv_src.type());
cv::filter2D(cv_src, cv_dst_original, CV_8U, cv_kernel, cv::Point(-1, -1), 0, cv::BORDER_REPLICATE);

// VisionWorks code

cv::Mat cv_dst_nvx(cv_src.size(), cv_src.type());

vx_image vx_src = nvx_cv::createVXImageFromCVMat(context, cv_src);
vx_image vx_dst = nvx_cv::createVXImageFromCVMat(context, cv_dst_nvx);

vx_uint32 conv_scale = 1024;

cv::Mat vx_kernel;
cv::flip(cv_kernel, vx_kernel, -1);
vx_kernel.convertTo(vx_kernel, CV_16S, conv_scale);

vx_convolution conv = vxCreateConvolution(context, vx_kernel.cols, vx_kernel.rows);
vxWriteConvolutionCoefficients(conv, vx_kernel.ptr<vx_int16>());
vxSetConvolutionAttribute(conv, VX_CONVOLUTION_ATTRIBUTE_SCALE, &conv_scale, sizeof(conv_scale));

vx_border_mode_t border;
border.mode = VX_BORDER_MODE_REPLICATE;
vxSetContextAttribute(context, VX_CONTEXT_ATTRIBUTE_IMMEDIATE_BORDER_MODE, &border, sizeof(border));

vxuConvolve(context, vx_src, conv, vx_dst);

vxReleaseImage(&vx_src);
vxReleaseImage(&vx_dst);
vxReleaseConvolution(&conv);

Integral Image

OpenCV cv::integral calculates exclusive sum as \( (W + 1) \times (H + 1) \) image:

\begin{eqnarray} sum(X,Y) &=& \sum_{x < X,y < Y} image(x,y) \end{eqnarray}

and uses the following formula to calculate sum over a rectangular region of the image:

\begin{eqnarray} \sum_{x_1 \leq x < x_2, y_1 \leq y < y_2} image(x,y) = sum(x_2, y_2) - sum(x_1, y_2) - sum(x_2, y_1) + sum(x_1, y_1) \end{eqnarray}

VisionWorks IntegralImage primitive calculates inclusive sum as \( W \times H \) image:

\begin{eqnarray} sum(X,Y) &=& \sum_{x \leq X,y \leq Y} image(x,y) \end{eqnarray}

and uses the following formula to calculate sum over a rectangular region of the image:

\begin{eqnarray} \sum_{x_1 \leq x < x_2, y_1 \leq y < y_2} image(x,y) = sum(x_2 - 1, y_2 - 1) - sum(x_1 - 1, y_2 - 1) - sum(x_2 - 1, y_1 - 1) + sum(x_1 - 1, y_1 - 1) \end{eqnarray}

Warping

VisionWorks WarpAffine and WarpPerspective primitives behave like cv::warpAffine and cv::warpPerspective with the cv::WARP_INVERSE_MAP flag. To port cv::warpAffine or cv::warpPerspective calls to VisionWorks, the transformation matrix must be transposed:

// x1 = a * x + b * y + c;
// y1 = d * x + e * y + f;

cv::Mat cv_src = cv::imread(fileName, cv::IMREAD_GRAYSCALE);

// OpenCV

float cv_mat_data[2][3] = {
    {a, b, c},
    {d, e, f}
};
cv::Mat cv_mat(2 /* rows */, 3 /* cols */, CV_32FC1, cv_mat_data);

cv::Mat cv_dst_original;
cv::warpAffine(cv_src, cv_dst_original, cv_mat, cv_src.size(), cv::INTER_LINEAR | cv::WARP_INVERSE_MAP);

// VisionWorks

vx_image vx_src = nvx_cv::createVXImageFromCVMat(context, cv_src);
vx_image vx_dst_1 = vxCreateImage(context, cv_src.cols, cv_src.rows, VX_DF_IMAGE_U8);

vx_float32 vx_mat_data[3][2] = {
    {a, d}, // 'x' coefficients
    {b, e}, // 'y' coefficients
    {c, f}, // 'offsets'
};
vx_matrix vx_mat_1 = vxCreateMatrix(context, VX_TYPE_FLOAT32, 2 /* cols */, 3 /* rows */);
vxWriteMatrix(vx_mat_1, vx_mat_data);

vxuWarpAffine(context, vx_src, vx_mat_1, VX_INTERPOLATION_TYPE_BILINEAR, vx_dst_1);

// or

vx_image vx_dst_2 = vxCreateImage(context, cv_src.cols, cv_src.rows, VX_DF_IMAGE_U8);

vx_matrix vx_mat_2 = nvx_cv::cloneCVMatToVXMatrix(context, cv_mat.t());

vxuWarpAffine(context, vx_src, vx_mat_2, VX_INTERPOLATION_TYPE_BILINEAR, vx_dst_2);

Find Homography

The VisionWorks FindHomography primitive returns the homography matrix in a format applicable for the VisionWorks WarpPerspective primitive. To use the homography matrix in OpenCV, it must be transposed:

vx_matrix vx_H = vxCreateMatrix(context, VX_TYPE_FLOAT32, 3, 3);
nvxuFindHomography(context, srcPoints, dstPoints, vx_H, NVX_FIND_HOMOGRAPHY_METHOD_RANSAC,
                   3.0f, 2000, 10, 0.995f, 0.45f, nullptr);

cv::Mat cv_H;
nvx_cv::copyVXMatrixToCVMat(vx_H, cv_H);
cv_H = cv_H.t();

cv::warpPerspective(cv_src, cv_dst, cv_H, cv_dst.size());