Using OpenCV Java with Eclipse

For some time OpenCV supports Java. This is a blessing, I have been waiting this for years. I am quite fond of Java, for such an old language they got lots of things right at the time (C# is better in a sense, they avoided the mistakes of Java). Anyway, here I will explain how to setup development environment for using OpenCV Java with Eclipse, so you can enjoy the benefits of garbage collected, very refactorable (rename variable, extract method and whatnot) modern language that enables you to write code with less effort and make less mistakes. Here we go.

First obtain a fresh release of OpenCV here and extract it under a simple location like C:\OpenCV-2.4.6\. I am using version 2.4.6, but the steps are more or less the same for other versions.

First we will define OpenCV as a user library in Eclipse, so we can reuse the configuration for any project. Launch Eclipse and select Window->Preferences from the menu.

Navigate under Java->Build Path->User Libraries and click New....

Enter a name, e.g. OpenCV-2.4.6, for your new library.

Now select your new user library and click "Add External JARs...".

Browse through C:\OpenCV-2.4.6\build\java\ and select opencv-246.jar. After adding the jar, extend the opencv-246.jar and select Native library location and press Edit....

Select External Folder... and browse to select the folder C:\OpenCV-2.4.6\build\java\x64. If you have a 32-bit system you need to select the x86 folder instead of x64.

Your user library configuration should look like this:

Now start creating a new Java project.

On the Java Settings step, under Libraries tab, select Add Library... and select OpenCV-2.4.6, then click Finish.

Libraries should look like this:

Now you have created and configured a new Java project it is time to test it. Create a new java file. Here is a starter code for your convenience:

import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.Mat;

public class Hello
{
 public static void main( String[] args )
 {
  System.loadLibrary( Core.NATIVE_LIBRARY_NAME );
  Mat mat = Mat.eye( 3, 3, CvType.CV_8UC1 );
  System.out.println( "mat = " + mat.dump() );
 }
}

When you run the code you should see 3x3 identity matrix as output.

That is it, enjoy your powerful, less painful development environment :)

How to build OpenCV library and debug it in VS2010?

Imagine you are writing some computer vision code using OpenCV in C++ and you are in one of your routine debugging sessions, the program is suspended on a line that is calling an OpenCV function. You want to step into the OpenCV function, you press F11 but the debugger steps over that line. In this article I am going to show how to build OpenCV from scratch and enable debugging OpenCV inside your own code.

If you are not interested in diving into OpenCV code and you do not wish to understand more about its mechanics you can just follow my Using OpenCV 2.4.x with Visual Studio 2010 tutorial.
Here is what we are going to do. To debug OpenCV we need pdb files, they don't come with the distribution. To obtain them we need to build the library from scratch.

First, download the latest CMake for Windows and install it.

After you launch CMake, show the path of your OpenCV installation as both source code and binary location (I suggest keeping that path simple).

As you can see I am using OpenCV 2.4.4 beta, I am choosing to put the build binaries to the same folder to keep everything together and simple.

After setting the paths, you should press "Configure" button. You will see a dialog asking for your compilers. Since I am compiling my code for 64 bit I select "Visual Studio 10 Win64". You can select "Visual Studio 10" if you are compiling for 32 bit. Make sure "Use default native compilers" radio button is selected. And press "Finish".

If you have Qt installed, this is a great opportunity to enable Qt, check the "WITH_QT" option. If you have no idea, just leave the options as they are, since building OpenCV with Qt is a topic of different article.

Now click "Configure" again. It will do some more work and "Generate" button next to "Configure" will be enabled, click it. It will generate solution and project files for Visual Studio 2010. We are done with CMake, you can close it now.

Navigate into your OpenCV folder, now you can see the generated files. Double click on "OpenCV.sln" to open it in VS2010. Select "Debug" configuration from the dropdown box (hopefully) located top of the VS window, under the menu:

Then click "Build Solution" under the "Build" menu (shortcut is F7). If everything goes well you should see a similar out:

Change the selection to "Release" from "Debug" and build again.

Congratulations you have successfully built OpenCV. Now dll and lib files are under [OPENCV_PATH]\bin and [OPENCV_PATH]\lib respectively.

Now open the project you were developing using OpenCV (assuming you already have some code that you were writing). Modify your project settings according to the new dll and lib paths. I am assuming you already know how to do that since you have a working project. You have to do this because the newly generated pdb files won't fit to dll and lib files that ships with the default distribution.

Next, under "Tools->Options" select "Debugging->Symbols".

Click to the folder icon and write the path to "bin" folder located under your OpenCV path inside the editable area (see figure above). Make sure that path is checked.

You are all done! Now put a break point on a line, and try to step into an OpenCV function!

Inside the infamous imshow() function

Using OpenCV 2.4.x with Visual Studio 2010 (tutorial)

Here in this tutorial I am going to show how to configure Visual Studio 2010 to use OpenCV 2.4.2 in your computer vision projects. Also, I am going to write a couple of lines of code to show new OpenCV C++ API.

By the time the tutorial was written the latest version of OpenCV was 2.4.2. The methods described below should, but may not, work for future versions. This tutorial targets computer vision course students at Computer Engineering in BOUN, but I will be generalizing as much as I can, so everyone can enjoy it.

Foreword

Before starting I would like to suggest a couple of things.

First, if you are ever going to write C++ code, read Effective C++, 3rd edition by Scott Meyers (CAUTION! 2nd edition is outdated, it is not worth reading imho). This book has many topics on how to write correct, working, easily maintainable C++ code. I see lots of boilerplate code around when it comes to C++, not only reading this book will reduce such practices, it will save you time. Besides, you will feel healthy mind-wise :) If there was to be only one book on C++, and the rest was to disappear, I would select this book. Be careful though, this book is *NOT* for beginners, you need to know a little bit of C++ to understand what it means.

For BOUN students who are taking Computer Vision course, the book is available in the library, also as an electronic resource.

Second, Google has really good guidelines on different programming languages, including C++. Although they are specialized for their internal use, I recommend following those guidelines. The guideline provide the motivation behind the rules, so it is also great place to learn how to write C++ properly. Again, it is not a place to learn C++, but to master C++.

Preparation

Make sure that you have Visual Studio 2010 (VS2010) installed.

Download OpenCV 2.4.2 for Windows and run the exe. Extract the contents to a folder. I prefer to put directly under root like "C:\OpenCV-2.4.2\".

Optionally, I recommend using Visual Assist X to write and refactor code easily.

Configuring VS2010

There are a couple of ways to configure Visual Studio 2010, I am going to show a way which I find convenient. We are going to configure it using property sheets. Whenever we need to create another project, we just need to add same property sheets to new project. Let's start.

Launch VS2010 and select "New project...".

Select "Empty Project", enter "hello-opencv" as Name, select a location to place project. I also prefer, for simplicity, not creating directory for solution so I unchecked that option. See the image below.

When done, press "OK".

Now we have created a project from scratch. It is time to link the libraries and make our project use OpenCV.

Switch to Property manager by clicking on the tab below. If you can't find the tab, you can access Property manager from menu View.

The default properties are for 32-bit systems (note the Win32 suffixes), because I am using 64-bit system, from now on I am going to describe the configuration for 64-bit systems.

From the "Build" menu, select "Configuration Manager". Under "Active solution platform" Win32 should be selected by default. Click on it and select "New...".

Select "x64" as the new platform and select Win32 to copy settings from. Make sure that "Create new project platforms" is checked.

Now your Property Manager view should look like this:

Select the Win32 configurations and delete them, you won't be needing those. Let me remind this one more time, these steps are only required if you are using 64-bit system and want to develop 64-bit applications.

To add a property sheet right click on "Debug | x64" and select "Add New Project Property Sheet". This property sheet will/should only include the details on how to link the OpenCV library to our project. Once this is done, the property sheet can be shared between projects on the same computer.

On the add new property sheet dialog, I entered "OpenCV-Debug" as the sheet name, because this sheet will link debug libraries. After pressing "Add" the new property sheet will show up in Property Manager. Double click on it, a dialog with lots of properties will be shown.

Under "Common Properties" -> "C/C++" -> "General", edit "Additional Include Directories" and add "C:\OpenCV-2.4.2\build\include". Of course don't forget to change the path accordingly if you have put OpenCV files to a different place. It is important that you select the "include" folder under "build" folder, not any other "include" folder.

Under "Common Properties" -> "Linker" -> "General", edit "Additional Library Directories" and add "C:\OpenCV-2.4.1\build\x64\vc10\lib". I won't tell you to change the path to your OpenCV path next time :) Here "x64" stands for 64-bit systems, if you are using 32-bit change it to x86. "vc10" stands for Visual C++ 2010, leave it as it is assuming you are using VS2010.

Under "Common Properties" -> "Linker" -> "Input", edit "Additional Dependencies" and add the following lib files:

opencv_calib3d242d.lib
opencv_contrib242d.lib
opencv_core242d.lib
opencv_features2d242d.lib
opencv_flann242d.lib
opencv_gpu242d.lib
opencv_haartraining_engined.lib
opencv_highgui242d.lib
opencv_imgproc242d.lib
opencv_legacy242d.lib
opencv_ml242d.lib
opencv_nonfree242d.lib
opencv_objdetect242d.lib
opencv_photo242d.lib
opencv_stitching242d.lib
opencv_ts242d.lib
opencv_video242d.lib
opencv_videostab242d.lib

Actually you won't probably need all the libraries. Linking the only necessary ones may make linking operation a little bit faster but for our case, that speed up is not worth the effort. Note the letter "d" in filenames just before the dot, that "d" stands for "Debug".

The steps are the same for "Release" configuration except the linked library filenames. You just need to get rid of the "d" letters before the dot. Debug libraries include additional instructions for debugging purposes such as variable watching, and they are not optimized. This makes debug libraries run slower. When you are dealing with lots of data or you just need more processing power you should switch to release configuration which is free of extra instructions and optimizes your code, hence runs much faster.

Hello OpenCV

OK. Now we are (almost) ready to write our first lines of code. Switch to "Solution Explorer" by clicking the tab below. Right click on "Source Files", select "Add" -> "New Item...". Select "C++ File" from the menu and type "main" as name.

Insert the following lines into the main.cpp and run the code (press F5).

#include <opencv2/opencv.hpp>

using namespace cv;

int main()
{
    Mat img(Mat::zeros(100, 100, CV_8U));

    imshow("window", img);

    waitKey();

    return 0;
}

You should get an error about a missing dll. Locate "hello-opencv.exe" in your project folder. Copy the dll files (preferably the ones that ends with letter "d") from "C:\OpenCV-2.3.1\build\x64\vc10\bin" to that folder.

By the way, using full namespaces like "using namespace cv;" is discouraged. I recommend using "using cv::Mat;", this will prevent name collisions and make the code more maintainable. See this discussion for details.

Some people prefer adding these dll into PATH environment variable, I prefer my projects to be self sufficient. It is good to be in control of which dll the project is using. If you are dealing with multiple projects using the different versions of the same library, if the dll names are the same, PATH approach will not work.

Run the code again. It should work fine and display a 100x100 black image in a window. If not that means there is a problem that you need to figure out. Try until you solve that problem.

Now modify the main.cpp so it is like:

#include <opencv2/opencv.hpp>

using namespace cv;

int main()
{
    Mat img;
    VideoCapture cap(0);

    while (true)
    {
        cap >> img;

        Mat edges;
        cvtColor(img, edges, CV_BGR2GRAY);

        Canny(edges, edges, 30, 60);

        imshow("window label", edges);
        waitKey(1);
    }

    return 0;
}

The code above needs a webcam connected to the computer, it captures images from a camera.

And run the code. If you get a missing "tbb_debug.dll" error, copy "C:\OpenCV-2.4.2\build\common\tbb\intel64\vc10\tbb_debug.dll" to the same folder under your project like other dlls (same directory as hello-opencv.exe).

Now run the code again. "Canny" is an edge detection algorithm, if the program runs well, you should see the detected edges of the captured image.

This completes our tutorial.

Closure

The documentation of OpenCV is quite good. Use it extensively.

You can follow OpenCV Tips for almost-daily tips on OpenCV.

I highly recommend using new OpenCV C++ API. The new API is clean, slick and well-designed. Avoid using old API, e.g. IplImage, unless you really know what you are doing. There are lots of information on internet about OpenCV which is outdated, beware of those. Always use the latest official documentation first.

PS. This tutorial is an updated version of OpenCV 2.3.1 installation tutorial. The contents are mostly same.

Using OpenCV 2.3.1 with Visual Studio 2010 (tutorial)

There is a newer version of this document: Using OpenCV 2.4.x with Visual Studio 2010

Here in this tutorial I am going to show how to configure Visual Studio 2010 to use OpenCV 2.3.1 in your computer vision projects. Also, I am going to write a couple of lines of code to show new OpenCV C++ API.

By the time the tutorial was written the latest version of OpenCV was 2.3.1. The methods described below should, but may not, work for different versions. This tutorial targets computer vision course students at Computer Engineering in BOUN, but I will be generalizing as much as I can, so everyone can enjoy it.

Foreword

Before starting I would like to suggest a couple of things.

First, if you are ever going to write C++ code, read Effective C++, 3rd edition by Scott Meyers (CAUTION! 2nd edition is outdated, it is not worth reading imho). This book has many topics on how to write correct, working, easily maintainable C++ code. I see lots of boilerplate code around when it comes to C++, not only reading this book will reduce such practices, it will save you time. Besides, you will feel healthy mind-wise :) If there was to be only one book on C++, and the rest was to disappear, I would select this book. Be careful though, this book is *NOT* for beginners, you need to know a little bit of C++ to understand what it means.

For BOUN students who are taking Computer Vision course, the book is available as an electronic resource in the library.

Second, Google has really good guidelines on different programming languages, including C++. Although they are specialized for their internal use, I recommend following those guidelines. The guideline provide the motivation behind the rules, so it is also great place to learn how to write C++ properly. Again, it is not a place to learn C++, but to master C++.

Preparation

Make sure that you have Visual Studio 2010 (VS2010) installed.

For BOUN students, it is available in license server. I don't quite know the details, you may need to work it on your own.

Download OpenCV 2.3.1 and extract the contents to a folder. I prefer to put directly under root like "C:\OpenCV-2.3.1\".

Optionally, I recommend using Visual Assist X to write and refactor code easily.

Configuring VS2010

There are a couple of ways to configure Visual Studio 2010, I am going to show a way which I find convenient. We are going to configure it using property sheets. Whenever we need to create another project, we just need to add same property sheets to new project. Let's start.

Launch VS2010 and select "New project...".

Select "Empty Project", enter "hello-opencv" as Name, select a location to place project. I also prefer, for simplicity, not creating directory for solution so I unchecked that option. See the image below.

When done, press "OK".

Now we have created a project from scratch. It is time to link the libraries and make our project use OpenCV.

Switch to Property manager by clicking on the tab below.

The default properties are for 32-bit systems (note the Win32 suffixes), because I am using 64-bit system, from now on I am going to describe the configuration for 64-bit systems.

From the "Build" menu, select "Configuration Manager". Under "Active solution platform" Win32 should be selected by default. Click on it and select "New...".

Select "x64" as the new platform and select Win32 to copy settings from. Make sure that "Create new project platforms" is checked.

Now your Property Manager view should look like this:

Select the Win32 configurations and delete them, you won't be needing those. Let me remind this one more time, these steps are only required if you are using 64-bit system and want to develop 64-bit applications.

To add a property sheet right click on "Debug | x64" and select "Add New Project Property Sheet". This property sheet will/should only include the details on how to link the OpenCV library to our project. Once this is done, the property sheet can be shared between projects on the same computer.

On the add new property sheet dialog, I entered "OpenCV-Debug" as the sheet name, because this sheet will link debug libraries. After pressing "Add" the new property sheet will show up in Property Manager. Double click on it, a dialog with lots of properties will be shown.

Under "Common Properties" -> "C/C++" -> "General", edit "Additional Include Directories" and add "C:\OpenCV-2.3.1\build\include". Of course don't forget to change the path accordingly if you have put OpenCV files to a different place. It is important that you select the "include" folder under "build" folder, not any other "include" folder.

Under "Common Properties" -> "Linker" -> "General", edit "Additional Library Directories" and add "C:\OpenCV-2.3.1\build\x64\vc10\lib". I won't tell you to change the path to your OpenCV path next time :) Here "x64" stands for 64-bit systems, if you are using 32-bit change it to x86. "vc10" stands for Visual C++ 2010, leave it as it is assuming you are using VS2010.

Under "Common Properties" -> "Linker" -> "Input", edit "Additional Dependencies" and add the following lib files:

opencv_calib3d231d.lib
opencv_contrib231d.lib
opencv_core231d.lib
opencv_features2d231d.lib
opencv_flann231d.lib
opencv_gpu231d.lib
opencv_haartraining_engined.lib
opencv_highgui231d.lib
opencv_imgproc231d.lib
opencv_legacy231d.lib
opencv_ml231d.lib
opencv_objdetect231d.lib
opencv_ts231d.lib
opencv_video231d.lib

Actually you won't probably need all the libraries. Linking the only necessary ones may make linking operation a little bit faster but for our case, that speed up is not worth the effort. Note the letter "d" in filenames just before the dot, that "d" stands for "Debug".

The steps are the same for "Release" configuration except the linked library filenames. You just need to get rid of the "d" letters before the dot. Debug libraries includes additional instructions for debugging purposes such as variable watching, and they are not optimized. This makes debug libraries run slower. When you are dealing with lots of data or you just need more processing power you should switch to release configuration which is free of extra instructions and optimizes your code, hence runs much faster.

Hello OpenCV

OK. Now we are (almost) ready to write our first lines of code. Switch to "Solution Explorer" by clicking the tab below. Right click on "Source Files", select "Add" -> "New Item...". Select "C++ File" from the menu and type "main" as name.

Insert the following lines into the main.cpp and run the code (press F5).

#include <opencv2/opencv.hpp>

using namespace cv;

int main()
{
    Mat img(Mat::zeros(100, 100, CV_8U));

    imshow("window", img);

    waitKey();

    return 0;
}

You should get an error about a missing dll. Locate "hello-opencv.exe" in your project folder. Copy the dll files (preferably the ones that ends with letter "d") from "C:\OpenCV-2.3.1\build\x64\vc10\bin" to that folder.

By the way, using full namespaces like "using namespace cv;" is discouraged. I recommend using "using cv::Mat;", this will prevent name collisions and make the code more maintainable. See this discussion for details.

Some people prefer adding these dll into PATH environment variable, I prefer my projects to be self sufficient. It is good to be in control of which dll the project is using. If you are dealing with multiple projects using the different versions of the same library, if the dll names are the same, PATH approach will not work.

Run the code again. It should work fine and display a 100x100 black image in a window. If not that means there is a problem that you need to figure out. Try until you solve that problem.

Now modify the main.cpp so it is like:

#include <opencv2 /opencv.hpp>

using namespace cv;

int main()
{
    Mat img;
    VideoCapture cap(0);

    while (true)
    {
        cap >> img;

        Mat edges;
        cvtColor(img, edges, CV_BGR2GRAY);

        Canny(edges, edges, 30, 60);

        imshow("window label", edges);
        waitKey(1);
    }

    return 0;
}

The code above needs a webcam connected to the computer, it captures images from a camera.

And run the code. You should get a missing "tbb_debug.dll" error. This is a bug of OpenCV-2.3.1. It should be shipped with tbb_debug.dll but it is missing in the package. That dll is the part of Threading Building Blocks project by Intel, you should download the latest version from here. The zip file that you have downloaded contains the dll, and you should copy that dll to the same folder under your project like other dlls (same directory as hello-opencv.exe).

Now run the code again. "Canny" is an edge detection algorithm, if the program runs well, you should see the detected edges of the captured image.

This completes our tutorial.

Closure

The documentation of OpenCV is quite good. Use it extensively.

I highly recommend using new OpenCV C++ API. The new API is clean, slick and well-designed. Avoid using old API, e.g. IplImage, unless you really know what you are doing. There are lots of information on internet about OpenCV which is outdated, beware of those. Always use the latest official documentation first.