Friday, December 11, 2015

Building JavaCPP presets for OpenCV 3 for Raspberry Pi (linux-arm)

Native image processing

To speed up image processing in a Java/Scala application on a Raspberry Pi, we resorted to 'opencv'. OpenCV already provides native Java binding. The disadvantage of this is however, that you manually must load those native libraries in your Java application.

JavaCV/JavaCPP to the rescue!

JavaCV is a wrapper using JavaCPP Presets like OpenCV.
JavaCPP provides a way to use OpenCV without manually adding code to load the native library. It does this via a static initialiser in the JavaCPP classes which makes sure the correct native library is loaded by the JVM. JavaCPP supports several C library, among which 'opencv'. They provide seperate jar files containing platform dependent native libraries, like for for macosx, linux-x86, linux-x86_64, windows-x86, windows-x86_64, android-arm, android-x86 (See JavaCPP presents in Maven Central). 

The advantage of JavaCV over the native OpenCV bindings, is that
- JavaCV combines all c-libraries in a single jar with a classifier for a specific platform
- It's easy to include platform specific dependencies in a project.
- JavaCV comes with a tool to automatically load the c-libraries from the jar.

However, they do not provide a 'linux-arm' build so you have to build it yourself. The easiest way, I think, is to build it on a real Pi. This can take several hours however.

Used resources:


If you just need the opencv JavaCPP preset then I provide these jars:
Note that these jars probably does not support video processing since I did not build opencv with any video dependency.

Note that JavaCPP 1.1 fixes a 'native-library-loading' issue on amd64 linux systems (for virtual machines, docker images etc) where on a 'amd64' architecture the native libs, of linux-x86_64 packages, did not get loaded.


Building for the Pi

JavaCPP comes with a 'cppbuild' script to build the opencv sources and create the java binding for it, but 'linux-arm' is not supported yet. In SNAPSHOT version there is some work in progress, but it seems this is only for cross compiling which does not support creating the java bindings currently.
So, only solution is to build it on the Pi itself.
The 'regular' opencv sources can be build for the Pi. Then these libs can be used to create the javacpp-opencv jar.

In short, these are the steps to take:

  • build opencv on pi (building the 'real' opencv project for pi is easier than trying to tweak the javacpp/opencv cppbuild script. Support for building linux-arm was added in SNAPSHOT but only for cross compilation. Does not work with Java wrappers, see below, and does not run on a real Pi.)
  • get opencv sources
  • run cmake to create native make files
  • run 'make -j5' to compile opencv on Pi. Use -j5 to use all cores!
  • run 'make install' to install libs in /usr/local/.. folders.
  • build javacpp for opencv using compiled opencv libs
  • make links to /usr/local's bin, include, lib and share folders in 'javacpp/opencv/cppbuild/linux-arm'.
  • build javacpp jar: 'mvn package'

Pitfalls:


  • make sure to checkout the same version by a git-tag in both 'opencv' and 'opencv_contrib' source folders.
  • run using 'screen' so build does not stop once you accidently disconnect.
  • ANT is required to be able to build java wrappers. Set both ANT_HOME and JAVA_HOME.
  • javacpp requires libraries in 'opencv_contrib' (e.g. 'face' module) so opencv build must include those modules.

Building OpenCV

sudo apt-get update
We did not need video libraries, but add those when you need them.
sudo apt-get install build-essential cmake pkg-config libpng12-0 libpng12-dev libpng++-dev libpng3 libpnglite-dev zlib1g-dbg zlib1g zlib1g-dev pngtools libtiff4-dev libtiff4 libtiffxx0c2 libtiff-tools libjpeg8 libjpeg8-dev libjpeg8-dbg libjpeg-progs 
sudo apt-get install screen
sudo apt-get install ant
in a screen session:
screen -S opencv
OpenCV Contrib needed because of 'face' dependency by javacpp
git clone https://github.com/Itseez/opencv_contrib.git
checkout correct version to build (same as opencv version!)
cd opencv_contrib
git checkout 3.0.0
cd ..
git clone https://github.com/Itseez/opencv.git
cd opencv
git checkout 3.0.0

ANT_HOME and JAVA_HOME needed to be able to build Java Wrappers and Java Tests.
It's assumed Java 8 is already installed on your Pi!
export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export ANT_HOME=/usr/share/ant
create dir for build into
mkdir build
cd build
All build options are mentioned in this presentation (slide 12): http://www.slideshare.net/andredsm/apresentacao-36089830
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D BUILD_EXAMPLES=D BUILD_PNG=ON -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules -D BUILD_opencv_face=ON -D BUILD_opencv_ximgproc=ON -D BUILD_opencv_optflow=ON ..
Check Java output is:
--   Java:
--     ant:                         /usr/bin/ant (ver 1.8.2)
--     JNI:                         /usr/lib/jvm/java-8-oracle/include /usr/lib/jvm/java-8-oracle/include/linux /usr/lib/jvm/java-8-oracle/include
--     Java wrappers:               YES
--     Java tests:                  YES
Start compiling:
make -j5 (aantal cores + 1)
Install bins, libs, includes, shares in /usr/local to be used by javacpp
sudo make install 

Building JavaCPP

Use natively build OpenCV libs.
JavaCPP needs Face.hpp !! --> is in opencv-contrib
cd  // to home dir
git clone https://github.com/bytedeco/javacpp-presets.git
cd javacpp-presets
checkout the version to build
git checkout 1.1
install main pom in repo (might not really be necessary)
mvn install -N 
make links in javacpp-presets/opencv/cppbuild/linux-arm to /usr/local/bin, /usr/local/share, /usr/local/lib, /usr/local/include
cd opencv/cppbuild/linux-arm
ln -s /usr/local/bin bin
ln -s /usr/local/include include
ln -s /usr/local/lib lib
ln -s /usr/local/share share
back to javacpp-presets/opencv folder
cd ../.. 
build javacpp library using previously build opencv libs
mvn clean package
Once completed, the 'target' folder contains a 'opencv-linux-arm.jar'
mv target/open-linux-arm.jar target/opencv-3.0.0-1.1-linux-arm.jar
Install jar in local repo or Nexus using these artifact details (also see 'target/maven-archive/pom.properties'):
groupId: org.bytedeco.javacpp-presets
artifactId: opencv
version: 3.0.0-1.1
classifier: linux-arm
packaging: jar

Now use this jar in your project using details as above.
Don't forget the classifier!!
<dependency>
<groupId>org.bytedeco.javacpp-presets</groupId>
<artifactId>opencv</artifactId>
<version>3.0.0-1.1</version>
<classifier>linux-arm</classifier>
</dependency>
or for SBT in build.sbt:
// Platform classifier for native library dependencies for javacpp-presets
lazy val platform = org.bytedeco.javacpp.Loader.getPlatform
in libraryDependencies:
"org.bytedeco" % "javacpp" % javacppVersion,
"org.bytedeco" % "javacv" % javacppVersion excludeAll(ExclusionRule(organization = "org.bytedeco.javacpp-presets")),
"org.bytedeco.javacpp-presets" % "opencv" % ("3.0.0-"+javacppVersion) classifier "",
"org.bytedeco.javacpp-presets" % "opencv" % ("3.0.0-"+javacppVersion) classifier platform,
"org.bytedeco.javacpp-presets" % "opencv" % ("3.0.0-"+javacppVersion) classifier "linux-arm",
in project/plugins.sbt
// `javacpp` are packaged with maven-plugin packaging, we need to make SBT aware that it should be added to class path.
classpathTypes += "maven-plugin"
// javacpp `Loader` is used to determine `platform` classifier in the project`s `build.sbt`
// We define dependency here (in folder `project`) since it is used by the build itself.
libraryDependencies += "org.bytedeco" % "javacpp" % "1.0"

Note on cross compiling:

OpenCV can also be cross compiled. Philipz's docker containers helps a lot setting up such an environment. The problem is however, you also want to build the java wrappers. Because the build architecture is set to 'arm' it will also look for an 'arm' version of the JVM and AWT libraries whereas you must use the native (ubuntu) java instead. Was not able to get a cross compile working yet.
Had to hack the /usr/local/cmake-2.8/Modules/FindJNI.cmake to make sure it found the 'amd64' java libraries (might be possible to set the JAVA_AWT_LIBRARY JAVA_JVM_LIBRARY vars instead) so cmake would make a build file where it would build the java wrappers and tests.
So, for now, this opencv-3.0.0-1.1-linux-arm.jar was still build on a real Pi (v2) and yes, it took hours to complete. ;-)