Static background subtraction using OpenCV

With background subtraction you can eliminate the background and focus on the actual object for further processing (detection, recogization, ...). Some algorythms provided in the OpenCV library are "learning" the background. The problem is if an object doesn't move, it will become the background as well. If you have a fixed camera - and so always the same background - you can just calculate the diffrence of two images.

Here's the code I used (you can download it here):

1:   #include <opencv2/core/core.hpp>   
2:   #include <opencv2/highgui/highgui.hpp>   
3:   #include <opencv2/imgproc/imgproc.hpp>   
4:   #include <iostream>   
5:   using namespace cv;   
6:   int main( int argc, char** argv )   
7:   {   
8:    // Read image given by user   
9:    Mat src = imread( "/users/christian/documents/programming/other/imgs/background.jpg", 0); // 1:color, 0:grayscale   
10:    Mat dst = imread("/users/christian/documents/programming/other/imgs/backtest.jpg", 0);   
11:    // background subtraction   
12:    Mat diff;   
13:    absdiff(src, dst, diff);   
14:    threshold(diff, diff, 10, 255, CV_THRESH_BINARY); // grayscale needed   
15:    // Show image in window   
16:    imshow("original", src);   
17:    imshow("new", dst);   
18:    imshow("diff", diff);   
19:    // Wait until user presses key   
20:    waitKey();   
21:    return 0;   
22:   }   

All it does is loading two images, calculating the differance, applying a threshold to highlight the object and displaying the 3 images.

This is just a theoretical example. Because here I use a static image to calculate the differnce, this method is very light sensitve. Lighting will also change the background and will no longer match exactly the stored background.

Program versions:

OS: Mac OS X 10.9.1
Xcode: 5.0.2
OpenCV: 2.4.8.0

Image processing with OpenCV

In the tutorial where I have explained how to write your first program, we already used some kind of image processing.

In this post I want to try out some other image processing possibilities.

Before we start we have to include some headers (from now on I will only use the newer headers from OpenCV 2):

 #include <opencv2/highgui/highgui.hpp>  
 #include <opencv2/imgproc/imgproc.hpp>  

After this I want to remind of the standard input and output operations that we will use.

Input and Output of images

Reading an image

We already did this in the tutorial. The command is:

 Mat image = imread( "PATH_TO_IMAGE", 1);  

So let's split this up:
Mat: this is the class for storing images, like int is for numbers
image: this is the name of the variable where the image is stored in
imread: command to read image
PATH_TO_IMAGE: this specifies the path to the image like "/users/christian/pictures/img.jpg"
1: specifies a color image with the 3 channels RGB. Change it to 0 for a grayscale image with just one channel.

Showing an image in a window

To do this we need the following command:

 imshow("Name of Window", image);

imshow: command to show image
Name of Window: this will be the name of the window where the image is shown in
image: variable of the image you want to display

Image Processing

I will just give you some examples. You can find all the possible commands in the OpenCV documentation. Let's start:

Allocation of channels

 vector<Mat>channels;  
 split(image, channels);  

Here we first create a variable called channels. With the command 'split' we split the RGB channels of the image and store them in channels. After that we can access each channel separately.

 imshow("Red", channels[2]);  
 imshow("Green", channels[1]);  
 imshow("Blue", channels[0]);  

Thresholding

Remember that every pixel of a image can be an integer in a range from 0 to 255. This means that a grayscale image can have 256 different scales of gray. The Picture below shows this

 threshold(image, image, 100, 255, CV_THRESH_BINARY);

image: the input and here as well the output image
100: this is the threshold; every pixel lower then 100 will be set to 0 (black)
255: all pixels over 100 will here be set to 255 (white)
CV_THRESH_BINARY: type of the threshold (doc)

For this I am using a grayscale image.

Blurring

 blur(image, image, Size (10, 10));  

image: the input and here as well the output image
Size: bluring kernel size, the higher the number - the more blur you'll get

Cutting

 Rect rect = Rect(100, 100, 200, 200); // Rectangle size 200x200, location x=100 y=100  
 image(rect).copyTo(imgaecut);         // Copy of the image  
 image(rect) *= 2;                     // Highlighting of the cut part in the original

Here we are creating a rectangle called rect with the size of 200x200 pixles and we locate it at x=y=100 pixels starting from the upper left corner. This part will be copied to a new image called imagecut. After this we highlight the cut part in the original image.

Don't forget to check the documentation for other processing commands!

Program versions:
OS: Mac OS X 10.9.1
Xcode: 5.0.2
OpenCV: 2.4.8.0

Reference: whydomath.org

Tutorial: First OpenCV program

After we installed OpenCV and configured Xcode it's time to create an easy program.

For the first program we want to refer to a OpenCV tutorial where we change the contrast and brightness of an image. But as I mentioned in my first post, this tutorial is outdated and was compiled with a older version of OpenCV. So we have to do some small changes to the code.

Edit the code

First edit the "main.cpp" file. Delete everything and copy the following code inside. You can also download it here.

1:  #include <opencv/cv.h>  
2:  #include <opencv/highgui.h>  
3:  #include <iostream>  
4:    
5:  using namespace cv;  
6:    
7:  double alpha; /**< Simple contrast control */  
8:  int beta; /**< Simple brightness control */  
9:    
10:  int main( int argc, char** argv )  
11:  {  
12:    // Read image given by user  
13:    // Change the path here to your image, look at form below  
14:    Mat image = imread( "/users/name/documents/programming/imgs/test1.jpg" );  
15:    Mat new_image = Mat::zeros( image.size(), image.type() );  
16:      
17:    // Initialize values  
18:    std::cout<<" Basic Linear Transforms "<<std::endl;  
19:    std::cout<<"-------------------------"<<std::endl;  
20:    std::cout<<"* Enter the alpha value [1.0-3.0]: ";std::cin>>alpha;  
21:    std::cout<<"* Enter the beta value [0-100]: "; std::cin>>beta;  
22:      
23:    // Do the operation new_image(i,j) = alpha*image(i,j) + beta  
24:    for( int y = 0; y < image.rows; y++ )  
25:    { for( int x = 0; x < image.cols; x++ )  
26:    { for( int c = 0; c < 3; c++ )  
27:    {  
28:      new_image.at<Vec3b>(y,x)[c] =  
29:      saturate_cast<uchar>( alpha*( image.at<Vec3b>(y,x)[c] ) + beta );  
30:    }  
31:    }  
32:    }  
33:      
34:    // Create Windows  
35:    namedWindow("Original Image", 1);  
36:    namedWindow("New Image", 1);  
37:      
38:    // Show stuff  
39:    imshow("Original Image", image);  
40:    imshow("New Image", new_image);  
41:      
42:    // Wait until user press some key  
43:    waitKey();  
44:    return 0;  
45:  }  

Note that I edited the include files in line 1 and 2. This is because of a newer version of OpenCV. If you navigate to /usr/local/include you will see that there are two folders now - opencv and opencv2. opencv contains the old header files and opencv2 the newer ones. So we just add the folder name to the path. Newer tutorials from the OpenCV website always include already the path and they use the opencv2 headers.

Run the program

Now it's time to finally compile and run the program. For this go to Product --> Run or just hit ⌘R

Down in the right corner you can see the console output where you can choose some values of the image manipulation.

Type in some numbers and hit enter. Here is the result for the values I chose.

Program versions:
OS: Mac OS X 10.9.1
Xcode: 5.0.2
OpenCV: 2.4.8.0

Tutorial: Configure Xcode for OpenCV programming

In this post I want to show you how to configure Xcode to work with OpenCV if you installed it like I explained here.

Start Xcode

1. When you open Xcode, select "Create a new Xcode project".

2. Under OS X Applications select "Command Line Tool".

3. Give your project a name and select as Type "C++".

4. Select a path where you want to save your project.

Configure Xcode

Before actually writing some code, we have to prepare Xcode so that it can use OpenCV. For this we have to include the header and library files.

As I explained in the install tutorial, OpenCV was installed to /usr/local. The header files are in /usr/local/include and the libraries are located in /usr/local/lib. These paths we have to add to the "Search Paths" in Xcode. To go there, you have to:

1. Select your project file in the project navigator on the left.

2. Now you can access the "Build Settings". In the search box type in "Search Paths" or simply scroll down until you see them.

3. Double click on "Header Search Paths" and add two lines:
    /usr/local/include
    /usr/local/lib

4. Double click on "Library Search Paths" and add this line:
    /usr/local/lib

5. Now we just have to add the *.dylib files from OpenCV. To do this:

  5.1 right click on your project file.

  5.2 Select "Add files to "projectname" ...".

  5.3 Don't select a path. Just hit / to open a line where you can specify a path. Enter /usr/local/lib and hit enter.

  5.4 Select all the *.dylib files you want to add. Just add all of them for now. If you want to you can create a folder and move all the files in there.

Finally it should look like in this screeshot below. With this configuration you are ready to write programs with OpenCV.

Remark:
In case you didn't build opencv yourself and installed it instead via macports, the procedure of configuring Xcode is the same. Just modify the paths. Instead of
/usr/local/lib and /usr/local/include
Macports installs the libraries in
/opt/local/lib and /opt/local/include

Program versions:

OS: Mac OS X 10.9.1
Xcode: 5.0.2
OpenCV: 2.4.8.0

Tutorial: Installing OpenCV on a Mac (OS X 10.9.1 Mavericks)

Since I found it quite hard to get OpenCV up and running and most of the tutorials I found were outdated, I will update them to work on the latest releases of OpenCV and Mac OS X. In every post I will also tell you which versions I used because I always thought this was a missing information. Here I used:

OS: Mac OS X 10.9.1
Xcode: 5.0.2
OpenCV: 2.4.8.0
MacPorts: 2.2.1
cmake: 2.8.12_3

OpenCV is available from various sources. You can get it via package managers like Homebrew or MacPorts or you can build it yourself from source. It's much easier if you get it from any of the mentioned package managers but in the beginning I got some errors with that when I tried to compile tutorials. Manly because it is installed in a different location as if you build it yourself. As the OpenCV tutorials always refer to the standard path, I would recommend for beginners to build it from source so you don't have to change the code of the examples and tutorials. So, let's go.

Preparation

Install Xcode

First of all you need to have Xcode installed. Just get it from the Mac App Store. The actual Version is 5.0.2. In other tutorials you will always find that you need to install the "Command Line Tools" for Xcode. Well, maybe you needed it. But Xcode 5 comes with these tools already installed. So nothing to do here.
Before you continue, launch Xcode. When you start it the first time you have to agree to the license agreement. If you don't do this, the following will not work.

Download OpenCV

Download the OpenCV Libraries. For Mac you want to use the Unix version. You will get a .zip file - unzip it.

Get MacPorts

Just download the package from their site MacPorts-2.2.1-10.9-Mavericks

... and install it. For help, refer to the MacPorts Guide.

You can check if it was installed correctly by typing

 port version  

in the terminal. The terminal app is in the Utilities folder in your applications.

Get cmake

For this you want to use the previously installed MacPorts. So launch your terminal app again. Then type this in:

 sudo port install cmake   

This may take a while until everything is downloaded and installed. You can check your installation by typing this in your terminal:

 port installed   

This will show all your installed ports. You should see cmake there.

Build OpenCV

We finally have made all the necessary preparations and are ready to build OpenCV.

1.
Copy the unpacked folder of OpenCV to your desktop und rename it to "opencv".

2.
Open your terminal and type the following in:

 cd Desktop/opencv  
 mkdir build  
 cd build  

With these commands you go to your OpenCV folder on the desktop, create a new folder inside named "build" and enter this folder.

3.
Now we can build and install OpenCV:

 cmake -G "Unix Makefiles" ..  
 make -j8  
 sudo make install  

That's it. You just built OpenCV from source and installed it to /usr/local.

In another post I will guide you through the process of creating your first OpenCV project in Xcode.


Source: Tilo Mitra