Using OpenCV on iPhone

Posted by Yoshimasa Niwa on 03/14, 2009

• 
• 

OpenCV is a library of computer vision developed by Intel, we can easily detect faces using this library for example. I’d note how to use it with iOS SDK, including the building scripts and a demo application. Here I attached screen shots from the demo applications.

Support Latest OpenCV and iOS SDK

Updated the project, support OpenCV 2.2.0, iOS SDK 4.3, Xcode 4 (Updated 04/17/2011.)

Getting Started

All source codes and resources are opened and you can get them from my github repository. It includes pre-compiled OpenCV libraries and headers so that you can easily start to test it. If you already have git command, just clone whole repository from github. If not, just take it by zip or tar from download link on github and inflate it.

% git clone git://github.com/niw/iphone_opencv_test.git

After getting source codes, open OpenCVTest.xcodeproj with Xcode, then build it. You will get a demo application on both iPhone Simulator and iPhone device.

Building OpenCV library from source code

• 

You can also make OpenCV library from source code using cross environment compile with gcc. I added some support script so that you can easy to do so. The important point is that iOS SDK doesn’t support dynamic link like “.framework”. We have to make it as static link library and link it to your application statically.

1. Building OpenCV requiers CMake. You can easily install it by using Homebrew orMacPorts.

   # Using Homebrew
   % brew install cmake
   # Using MacPorts
   % sudo port install cmake -gui
   

   If you’ve already installed recent Java update, you may be asked to installjavadeveloper_10.6_10m3261.dmg. This is weird but cmake needs jni.h which is removed from recent Java update, you can download it from here for Mac OS X 10.6 which may require you to subscribe Apple Developer Connection. Yes, Apple is now going to deprecate Java on MacOS X(Updated 10/30/2010).

2. Getting source code from sourceforge. I tested with OpenCV-2.2.0.tar.bz2.

3. Extract downloaded archive on the top of demo project directory

   % tar xjvf OpenCV-2.2.0.tar.bz2
   

4. Apply patch for iOS SDK

   % cd OpenCV-2.2.0
   % patch -p1 < ../OpenCV-2.2.0.patch
   

5. Following next steps to build OpenCV static library for simulator. All files are installed intoopencv_simulator directory. When running make command, you’ve better assign -j option and number according to number of your CPU cores. Without -j option, it takes a long time.

   % cd .. # Back to the top of demo project directory.
   % mkdir build_simulator
   % cd build_simulator
   % ../opencv_cmake.sh Simulator ../OpenCV-2.2.0
   % make -j 4
   % make install
   

6. Following next steps to build OpenCV static library for device All files are installed intoopencv_device directory.

   % cd .. # Back to the top of demo project directory.
   % mkdir build_device
   % cd build_device
   % ../opencv_cmake.sh Device ../OpenCV-2.2.0
   % make -j 4
   % make install
   

Build support script

Build support script opencv_cmake.sh has some options to build OpenCV with iOS SDK. Try --helpoption to get the all options of it.

Converting images between UIImage and IplImage

OpenCV is using IplImage structure for processing, and iOS SDK using UIImage object to display it on the screen. This means, we need a converter between UIImage and IplImage. Thankfully, we can do with iOS SDK APIs.

Creating IplImage from UIImage is like this.

// NOTE you SHOULD cvReleaseImage() for the return value when end of the code.
- (IplImage *)CreateIplImageFromUIImage:(UIImage *)image {
  // Getting CGImage from UIImage
  CGImageRef imageRef = image.CGImage;

  CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
  // Creating temporal IplImage for drawing
  IplImage *iplimage = cvCreateImage(
    cvSize(image.size.width,image.size.height), IPL_DEPTH_8U, 4
  );
  // Creating CGContext for temporal IplImage
  CGContextRef contextRef = CGBitmapContextCreate(
    iplimage->imageData, iplimage->width, iplimage->height,
    iplimage->depth, iplimage->widthStep,
    colorSpace, kCGImageAlphaPremultipliedLast|kCGBitmapByteOrderDefault
  );
  // Drawing CGImage to CGContext
  CGContextDrawImage(
    contextRef,
    CGRectMake(0, 0, image.size.width, image.size.height),
    imageRef
  );
  CGContextRelease(contextRef);
  CGColorSpaceRelease(colorSpace);

  // Creating result IplImage
  IplImage *ret = cvCreateImage(cvGetSize(iplimage), IPL_DEPTH_8U, 3);
  cvCvtColor(iplimage, ret, CV_RGBA2BGR);
  cvReleaseImage(&iplimage);

  return ret;
}

Don’t forget release IplImage after using it by cvReleaseImage!

And creating UIImage from IplImage is like this.

// NOTE You should convert color mode as RGB before passing to this function
- (UIImage *)UIImageFromIplImage:(IplImage *)image {
  CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
  // Allocating the buffer for CGImage
  NSData *data =
    [NSData dataWithBytes:image->imageData length:image->imageSize];
  CGDataProviderRef provider =
    CGDataProviderCreateWithCFData((CFDataRef)data);
  // Creating CGImage from chunk of IplImage
  CGImageRef imageRef = CGImageCreate(
    image->width, image->height,
    image->depth, image->depth * image->nChannels, image->widthStep,
    colorSpace, kCGImageAlphaNone|kCGBitmapByteOrderDefault,
    provider, NULL, false, kCGRenderingIntentDefault
  );
  // Getting UIImage from CGImage
  UIImage *ret = [UIImage imageWithCGImage:imageRef];
  CGImageRelease(imageRef);
  CGDataProviderRelease(provider);
  CGColorSpaceRelease(colorSpace);
  return ret;
}

Ok, now you can enjoy with OpenCV with iPhone!

Using OpenCV library in your own project

The demo application which you can download from my repository is well configured to use these libraries. If you wanted to use OpenCV libraries on your own project, you should need to adding next configurations on it. You can see these settings on the Xcode project of this demo application.

• Add libopencv_core.a etc, from OpenCV lib directory for either simulators or devices. Actually Xcode doesn’t care which one is for devices or simulators at this point because it is selected by the library search path.
• Add Accelerate.framework which is used internally from OpenCV library.
• Select your active build target, then open the build tab in the info panel by Get Info menu.
  • Add -lstdc++ and -lz to Other Linker Flags
  • Add libz.dylib to your project.
  • Add path to OpenCV include directory to Header Search Paths for both simulators and devices.
  • Add path to OpenCV lib directory to Library Search Paths for both simulators and devices.

Change Log

• 04/17/2011 - Support OpenCV 2.2.0 + iOS SDK 4.3 + Xcode 4, Thank you for all your comments!
• 10/30/2010 - Updates for recent changes on the repository and the development environment, Thank you for your comments!
• 08/22/2010 - Support OpenCV 2.1.0 + iOS SDK 4.0
• 12/21/2009 - Support Snow Leopard + iPhone SDK 3.1.2, Thank you Hyon!
• 11/15/2009 - Support OpenCV 2.0.0 + iPhone SDK 3.x
• 03/14/2009 - Release this project with OpenCV 1.0.0 + iPhone SDK 2.x