What is OpenCV?

OpenCV stands for Open-Source Computer-Vision Library.

As name suggests, library has lots of various features for computer vision, for instance:

OpenCV is cross-platform, supporting major desktop platforms (Windows, Linux, MacOS), as well as mobile platforms (iOS, Android).

The project is open-source software, licensed under BSD license.

Also, it has bindings for Java, Python, Haskel, MATLAB, etc.

OpenCV has strong focus on performance, with optimized code for various microarchitectures (SSE and AVX on x86, NEON on ARM, VSX on PowerPC, etc).

Library is suitable for heterogeneous computing, supporting CUDA and OpenCL.

To summarize, OpenCV library is very large, and has tons of features for the computer vision. In addition, it has collection of additional modules called OpenCV Contrib, if base is not enough.

What’s new in OpenCV 4.0.0 release?

OpenCV has released final version 4.0.0 on November 2018.

There are a lot of new features (see the complete changelog), for instance:

also, OpenCV 4.x is getting rid of some technical debt, for example:

as usual, there are numerous bug fixes and performance improvements.

OpenCV G-API

Let’s take a deeper look at new OpenCV feature called G-API (stands for Graph API).

Previously, with classic OpenCV 2.x API, programming model was very traditional - you call OpenCV functions, they perform some computations and return the result. This model should look familar and natural to the most programmers, as it’s similar to regular filesystem API, for example. OpenCV 4.x introduces very different programming model where you define pipeline of operations to be performed first, and then apply this pipeline to some actual data. In other words, whenever you call OpenCV G-API function, execution is deferred (lazily-evaluated), and deferred operation result is being returned instead of actual computation result. This concept might sound very similar to Ranges TS (or Range V3, or Boost Range).

This is an especially important feature, as if you don’t need to get an intermediate results, you can easily off-load the entire pipeline into the GPU, therefore there will be no intermediate copying from the system to the video memory and vice versa - only initial input and final output has to be loaded to/from the GPU.

Practicing G-API

Complete code examples from this blog post are available on GitHub: opencv4-demo (project uses CMake and conan to build).

For instance, we have the following code which uses classic OpenCV 2.x API:

#include <cstdlib>
#include <opencv2/imgproc.hpp>
#include <opencv2/imgcodecs.hpp>

int main(int argc, char * argv[])
{
    cv::Mat imgIn = cv::imread("in.png"), imgBlur, imgGray, imgOut, sobelX, sobelY, gradX, gradY;

    cv::GaussianBlur(imgIn, imgBlur, cv::Size(3, 3), 0, 0, cv::BORDER_DEFAULT);

    cv::cvtColor(imgBlur, imgGray, cv::COLOR_BGR2GRAY);

    cv::Sobel(imgGray, sobelX, CV_16S, 1, 0, 3);
    cv::Sobel(imgGray, sobelY, CV_16S, 0, 1, 3);

    cv::convertScaleAbs(sobelX, gradX);
    cv::convertScaleAbs(sobelY, gradY);

    cv::addWeighted(sobelX, 0.5, sobelY, 0.5, 0, imgOut);

    cv::imwrite("out.png", imgOut);

    return EXIT_SUCCESS;
}

The example takes an input image file, blurs it, converts to the grayscale and finally applies the Sobel operator, then saves result of operator applied to an another image file. So, if we have the following image file as an input:

A color picture of a steam engine

then result might look like:

The Sobel operator applied to that image

Such code is usually used for the edge detection, which is frequently used image processing task.

Okay, how to migrate the code you already have to the G-API? First off, you’ll need to include additional OpenCV headers:

#include <opencv2/gapi.hpp>
#include <opencv2/gapi/core.hpp>
#include <opencv2/gapi/imgproc.hpp>

Second, functions from the cv:: namespace are being replaced by corresponding functions from the new cv::gapi:: namespace, e.g. cv::Sobel becomes cv::gapi::Sobel, and so on. These functions do not have an output argument, instead, they return value of cv::GMat type (analogue of well-known cv::Mat type). Same for the input arguments - they also accept cv::GMat. Some functions might be missing in the cv::gapi:: namespace, for instance cv::convertScaleAbs, but it’s pretty straightforward to implement it yourself:

static cv::GMat convertScaleAbs(const cv::GMat & src, double alpha = 1.0, double beta = 0.0)
{
    auto result = cv::gapi::absDiffC(cv::gapi::addC(cv::gapi::mulC(src, alpha), beta), 0.0);
    return cv::gapi::convertTo(result, CV_8UC1);
}

With implementation above, rewriting image processing code is pretty straightforward:

    auto imgBlur = cv::gapi::gaussianBlur(gIn, cv::Size(3, 3), 0, 0, cv::BORDER_DEFAULT);

    auto imgGray = cv::gapi::convertTo(imgBlur, CV_32F);

    auto sobelX = cv::gapi::Sobel(imgGray, CV_16S, 1, 0, 3);
    auto sobelY = cv::gapi::Sobel(imgGray, CV_16S, 0, 1, 3);

    auto gradX = convertScaleAbs(sobelX);
    auto gradY = convertScaleAbs(sobelY);

    auto gOut = cv::gapi::addWeighted(sobelX, 0.5, sobelY, 0.5, 0);

Once pipeline is established, it’s time to construct the cv::GComputation object:

    cv::GComputation computation(cv::GIn(gIn), cv::GOut(gOut));

And finally, computation might be applied to the actual data:

    computation.apply(cv::gin(imgIn), cv::gout(imgOut));

At this time, actual data processing takes its place.

The complete example using the G-API:

#include <cstdlib>
#include <opencv2/imgproc.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/gapi.hpp>
#include <opencv2/gapi/core.hpp>
#include <opencv2/gapi/imgproc.hpp>

static cv::GMat convertScaleAbs(const cv::GMat & src, double alpha = 1.0, double beta = 0.0)
{
    auto result = cv::gapi::absDiffC(cv::gapi::addC(cv::gapi::mulC(src, alpha), beta), 0.0);
    return cv::gapi::convertTo(result, CV_8UC1);
}

int main(int argc, char * argv[])
{
    cv::GMat gIn;

    auto imgBlur = cv::gapi::gaussianBlur(gIn, cv::Size(3, 3), 0, 0, cv::BORDER_DEFAULT);

    auto imgGray = cv::gapi::convertTo(imgBlur, CV_32F);

    auto sobelX = cv::gapi::Sobel(imgGray, CV_16S, 1, 0, 3);
    auto sobelY = cv::gapi::Sobel(imgGray, CV_16S, 0, 1, 3);

    auto gradX = convertScaleAbs(sobelX);
    auto gradY = convertScaleAbs(sobelY);

    auto gOut = cv::gapi::addWeighted(sobelX, 0.5, sobelY, 0.5, 0);

    cv::GComputation computation(cv::GIn(gIn), cv::GOut(gOut));

    cv::Mat imgIn = cv::imread("in.png"), imgOut;

    computation.apply(cv::gin(imgIn), cv::gout(imgOut));

    cv::imwrite("out.png", imgOut);

    return EXIT_SUCCESS;
}

Conclusion

OpenCV 4.0 release adds very foundational changes, that completely change the way how do you write programs, making support of heterogeneous computing much more straightforward. Feel free to try to experiment with new OpenCV features, such as G-API, check out the opencv4-demo repository, in order to compile and run examples from this article.