Getting the OpenCV tutorial-4-opencl android app compiling and working on MacOS

I’ve been unable to get opencv working at all in Android Studio, but here’s a (slightly painful) way of getting it working on MacOS, at least as of the December, 2017.

First, current android sdk tools (ie ‘android’) will not work. You’ll have to download and use an old version. Assuming your sdk is installed in ~/Library/Android/sdk, run this on the terminal:

cd ~/Library/Android/sdk
mv tools tools.orig
curl -O https://dl.google.com/android/repository/tools_r25.2.3-macosx.zip
unzip tools_r25.2.3-macosx.zip
rm tools_r25.2.3-macosx.zip

Now we grab all of the sources we need to build:

export BUILD_HOME=$HOME/opencv    # change as you would like
mkdir -p $BUILD_HOME
cd $BUILD_HOME

# change these if they're not correct
export ANDROID_NDK=~/android-ndk-r11c
export ANDROID_SDK=~/Library/Android/sdk

# Download the necessary code
git clone git@github.com:opencv/opencv.git
mv opencv opencv-3.3.0
git clone git@github.com:opencv/opencv_contrib.git

# Creating our custom OpenCL SDK
export OPENCL_SDK=$BUILD_HOME/OpenCLSDK
mkdir $OPENCL_SDK && cd $OPENCL_SDK
mkdir lib include
# Copy the OpenCL headers used to compile OpenCV 
cp -r $BUILD_HOME/opencv-3.3.0/3rdparty/include/opencl/1.2/CL include/
# Download the multiplatform version of the C++ OpenCL Wrapper
cd include/CL/ && mv cl.hpp cl.hpp.orig && wget https://www.khronos.org/registry/cl/api/2.1/cl.hpp && cd ../..

# Grab the opencl stub and copy the c and h files into place
cd $BUILD_HOME
git clone https://github.com/krrishnarraj/libopencl-stub.git
cp libopencl-stub/include/libopencl.h opencv-3.3.0/samples/android/tutorial-4-opencl/jni/
cp libopencl-stub/src/libopencl.c opencv-3.3.0/samples/android/tutorial-4-opencl/jni/

# Set the folder where we are going to compile opencv, it will be used when we build the project
export OPENCV_ANDROID_SDK=$BUILD_HOME/build
mkdir -p $OPENCV_ANDROID_SDK
cd $OPENCV_ANDROID_SDK

# need to add the opencl stub to the build and remove the OpenCL library

At this point you’re almost good to start building, but we need to change one thing. The build by default uses a libOpenCL.so that we’re supposed to pull from the device, and then later link with the app and install back on the device. Since this library is device specific, its kind of a bad idea. (TM) Instead, we downloaded opencl stubs that dynamically open the appropriate libopencl.so library that already exists on the device and uses that. In theory, that means we should be able to use it on any device that has an opencl library installed. (Which should be pretty much all devices that do support opencl.)

The problem is we need to remove the opencl library, and add the libopencl.c file to the appropriate make-ish file. You do this by editing $BUILD_HOME/opencv-3.3.0/samples/android/tutorial-4-opencl/jni/Android.mk, removing the ‘-lOpenCL’, and adding ‘libopencl.c’ to the LOCAL_SRC_FILES line.

Now you’re ready for the first whack at building:

cd $OPENCV_ANDROID_SDK
cmake -Wno-dev \
-DCMAKE_TOOLCHAIN_FILE=$BUILD_HOME/opencv-3.3.0/platforms/android/android.toolchain.cmake \
-DANDROID_ABI="armeabi-v7a with NEON" \
-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-DBUILD_ANDROID_EXAMPLES=ON \
-DINSTALL_ANDROID_EXAMPLES=ON \
-DWITH_OPENCL=YES \
-DANDROID_OPENCL_SDK=$OPENCL_SDK \
-DOPENCV_EXTRA_MODULES_PATH=$BUILD_HOME/opencv_contrib/modules \
-DANDROID_NATIVE_API_LEVEL=14 \
-DANDROID_SDK_TARGET=24 \
$BUILD_HOME/opencv-3.3.0
make -j8

This will appear to be doing everything you want until you get to what we’re interested in, the tutorial-4-opencl build. There, it fails:

ld: error: cannot find -lOpenCL
(and then a bunch more errors because of that)

We’re still getting the OpenCL.so error because we have it listed in opencv-3.3.0/samples/android/tutorial-4-opencl/CMakeLists.txt but if we remove the -lOpenCL from there, for some reason the libopencv.so isn’t included in the
tutorial-4-opencl build and the app crashes when it can’t open it. So instead, we have to remove the -lOpenCL from the samples/android/tutorial-4-opencl/CMakeFiles/JNIpart.dir/link.txt file, and run ‘make -j8’ again.

Now everything should have built without errors. You should be able to install the app on your android with:

$ANDROID_SDK/platform-tools/adb install samples/android/tutorial-4-opencl/.build/bin/example-tutorial-4-opencl-debug.apk

If anyone figures out how to skip the double compile stage, please let me know!

Using non-secure web services in iOS

As of iOS7ish, if you try to access a normal web site with http:// instead of the more secure https://, you get an error at runtime:

App Transport Security has blocked a cleartext HTTP (http://) resource load since it is insecure. Temporary exceptions can be configured via your app’s Info.plist file. To allow any http:// to get through, add the following to your Info.plist file

	<key>NSAppTransportSecurity</key>
	<dict>
		<key>NSAllowsArbitraryLoads</key>
		<true/>
	</dict>

or if you want specific domains to be allowed (example.com in this case):

	<key>NSAppTransportSecurity</key>
	<dict>
		<key>NSExceptionDomains</key>
		<dict>
			<key>example.com</key>
			<dict>
				<key>NSIncludesSubdomains</key>
				<true/>
				<key>NSTemporaryExceptionAllowsInsecureHTTPLoads</key>
				<true/>
				<key>NSTemporaryExceptionMinimumTLSVersion</key>
				<string>TLSv1.1</string>
			</dict>
		</dict>
	</dict>

That being said, its better to use secure web sites when possible.

Restricting Orientations in different view controllers

At some point you may want to restrict certain device orientations to certain view controllers. Its quite easy to do from your App Delegate.

First add a new variable to your AppDelegate.swift file:

   var allowableOrientations: UIInterfaceOrientationMask = .All

Next, implement the application:supportedInterfaceOrientationsForWindow function in the same file:

func application(application: UIApplication, supportedInterfaceOrientationsForWindow window: UIWindow?) -> UIInterfaceOrientationMask {
    return allowableOrientations;
}

This will be called whenever you rotate the device to determine if that orientation is allowed. It doesn’t, however, force a rotation. We can implement another function in the same file to handle setting the allowableOrientations file and making sure the current orientation is one of those:

    func setAllowableOrientations(orientationMask:UIInterfaceOrientationMask,
                               defaultOrientation:UIInterfaceOrientation) {
        self.allowableOrientations=orientationMask;
        if ((orientationMask.rawValue & UInt(UIDevice.currentDevice().orientation.rawValue)) == 0) {
            UIDevice.currentDevice().setValue(defaultOrientation.rawValue, forKey: "orientation")
        }
    }

To use it, simply call this function from your View Controller’s viewDidLoad function, similar to this:

    (UIApplication.sharedApplication().delegate as! AppDelegate).setAllowableOrientations(.Landscape,defaultOrientation: .LandscapeRight)

That’s it!

Exporting a Development-signed OSX Application that uses Maps

If you want someone to test your OSX application that uses Maps, and it keep crashing on their Mac, it may be because they need their Mac to be in developer mode. You can see if its currently in developer mode with the following terminal command:

DevToolsSecurity -status

If you want to enable developer mode, use:

DevToolsSecurity -enable

And if for some reason you want to disable developer mode, use:

DevToolsSecurity -disable

You’ll most likely also have to right click on the app, and select ‘Open’ (and confirm) to run the app the first time.

Making a bootable Mac Install ISO

In terminal, change into the ‘Install ‘ folder in your Applications folder after downloading it. Then paste this into a file called makebootableiso.sh:

#!/bin/bash  
  
hdiutil attach ./Contents/SharedSupport/InstallESD.dmg -noverify -nobrowse -mountpoint /Volumes/install_app  
hdiutil convert /Volumes/install_app/BaseSystem.dmg -format UDSP -o ./OSXInstall  
hdiutil resize -size 8g ./OSXInstall.sparseimage  
hdiutil attach ./OSXInstall.sparseimage -noverify -nobrowse -mountpoint /Volumes/install_build  
rm /Volumes/install_build/System/Installation/Packages  
cp -rp /Volumes/install_app/Packages /Volumes/install_build/System/Installation/  
cp -rp /Volumes/install_app/BaseSystem.chunklist /Volumes/install_build  
cp -rp /Volumes/install_app/BaseSystem.dmg /Volumes/install_build  
hdiutil detach /Volumes/install_app  
hdiutil detach /Volumes/install_build  
hdiutil resize -size `hdiutil resize -limits ./OSXInstall.sparseimage | tail -n 1 | awk '{ print $1 }'`b ./OSXInstall.sparseimage  
hdiutil convert ./OSXInstall.sparseimage -format UDTO -o ./OSXInstall  
rm ./OSXInstall.sparseimage  
mv ./OSXInstall.cdr ~/Desktop/OSXInstall.iso  

Finally, run ‘sudo sh makebootableiso.sh’ and let it go. You should end up with a ‘OSXInstall.iso’ file on your Desktop. Burn that or use it in a VM to install OSX!

ERROR iTMS-90022: Missing required icon file. (120×120)

If you get a iTMS-90022 error message and you have what looks like the proper 120×120 icon file in your Images.xcassets, you can try to get this working the old fashioned way by adding the appropriate .png files to your project and updating your Info*.plist file. The entries for the plist file for iPhone targets should be:

<key>CFBundleIconFiles</key>
<array>
    <string>Icon-Small</string>
    <string>Icon-Small-40</string>
    <string>Icon-Small-50</string>
    <string>Icon</string>
    <string>Icon-60</string>
    <string>Icon-72</string>
</array>

Make sure your png files (along with the @2, @3 versions) match this naming convention as well. For me, it was the ‘Spotlight’ files that were misnamed to ‘Icon-40.png’, etc instead of ‘Icon-Small-40.png’.

The entries for the plist file for iPad targets should be:

<key>CFBundleIconFiles~ipad</key>
<array>
    <string>Icon-Small</string>
    <string>Icon-Small-40</string>
    <string>Icon-Small-50</string>
    <string>Icon-72</string>
    <string>Icon-76</string>
</array>

If you have a universal app, add both to your plist file, and make sure the matching png files are all in your project as well.

More info here if you are a developer.

Saving UIImage to the photo album.

The simplest way to save a UIImage to the photo album is with the UIImageWriteToSavedPhotosAlbum() function. The simplest way is by specifying the UIImage, and none of the other parameters:

    UIImageWriteToSavedPhotosAlbum(image, nil,nil, nil);

However, this will save the image as a jpg, which is not always ideal. If you want to save it as PNG, you have to first encode the image as a PNG, then make a new image based on that PNG so that the PNG data is attached to it, and then save the new image:

    NSData *data=UIImagePNGRepresentation(image);
    image=[UIImage imageWithData:data];
    UIImageWriteToSavedPhotosAlbum(image, nil,nil, nil);

You can also force JPEG format (with your own compression quality) by doing the same thing, but with the UIImageJPEGRepresentation() call:

    NSData *data=UIImageJPEGRepresentation(image,1.0); // best quality
    image=[UIImage imageWithData:data];
    UIImageWriteToSavedPhotosAlbum(image, nil,nil, nil);

Making your iOS OpenGLES view transparent

At some point you may want to make your OpenGL ES view transparent, so you can display something else behind it. (Maybe a camera view?) For this article, I’m going to assume you are using a Game project template from XCode 6.

In your GameViewController.m’s viewDidLoad function, just before the ‘[self setupGL];’ call, add the following lines:

    view.backgroundColor=[UIColor clearColor];
    
    CAEAGLLayer *eaglLayer=(CAEAGLLayer*)self.view.layer;
    eaglLayer.opaque=NO;

Also, in the glkView:drawInRect function, change the glClearColor line to be all zeroes:

    glClearColor(0.00f, 0.00f, 0.00f, 0.0f);

That should be it.

CLLocationManager not working in iOS8?

iOS8 has added some security measures to their GPS location services to make sure the user gives permission to track the device. First, you need to add a NSLocationWhenInUseUsageDescription String entry into your Info.plist file. This contains the text you want to display to the user when they are asked for permission to grab their GPS location. If you right click on Info.plist and ‘Open As->Source Code’, add something similar to this:

    <key>NSLocationWhenInUseUsageDescription</key>
    <string>We need to use your GPS to determine your location.</string>

You also need to make sure the user is asked for permission with the requestWhenInUseAuthorization call. This will only work in viewWillAppear: or later. Its a good idea to initialize the locationManager in viewDidLoad: so your code may look like this:

- (void)viewDidLoad {
    [super viewDidLoad];

    _locationManager = [[CLLocationManager alloc] init];
    _locationManager.delegate = self;
    _locationManager.desiredAccuracy = kCLLocationAccuracyBest;
    [_locationManager startUpdatingLocation];
}

- (void)viewWillAppear:(BOOL)animated {
    [super viewWillAppear:animated];

    // for iOS8 request authorization
    if ([_locationManager respondsToSelector:@selector(requestWhenInUseAuthorization)]) {
        [_locationManager requestWhenInUseAuthorization];
    }
}

With those two changes, your app should again be retrieving gps locations. Background locations are handled in another (but similar) way.