Native image processingTo speed up image processing in a Java/Scala application on a Raspberry Pi, we resorted to 'opencv'. OpenCV already provides native Java binding. The disadvantage of this is however, that you manually must load those native libraries in your Java application.
JavaCV/JavaCPP to the rescue!JavaCV is a wrapper using JavaCPP Presets like OpenCV.
JavaCPP provides a way to use OpenCV without manually adding code to load the native library. It does this via a static initialiser in the JavaCPP classes which makes sure the correct native library is loaded by the JVM. JavaCPP supports several C library, among which 'opencv'. They provide seperate jar files containing platform dependent native libraries, like for for macosx, linux-x86, linux-x86_64, windows-x86, windows-x86_64, android-arm, android-x86 (See JavaCPP presents in Maven Central).
The advantage of JavaCV over the native OpenCV bindings, is that
- JavaCV combines all c-libraries in a single jar with a classifier for a specific platform
- It's easy to include platform specific dependencies in a project.
- JavaCV comes with a tool to automatically load the c-libraries from the jar.
- JavaCPP Presets Readme: https://github.com/bytedeco/javacpp-presets/blob/master/README.md
- Installing opencv 3 and javacv on raspberry pi: http://alltechanalysis.blogspot.nl/2015/09/installing-opencv-30-and-javacv-on.html
- Using JavaCV in the raspberry pi linux arm: http://salaboy.com/2013/06/14/using-javacv-in-the-raspberry-pi-linux-arm/
If you just need the opencv JavaCPP preset then I provide these jars:
Note that these jars probably does not support video processing since I did not build opencv with any video dependency.
Note that JavaCPP 1.1 fixes a 'native-library-loading' issue on amd64 linux systems (for virtual machines, docker images etc) where on a 'amd64' architecture the native libs, of linux-x86_64 packages, did not get loaded.
Building for the PiJavaCPP comes with a 'cppbuild' script to build the opencv sources and create the java binding for it, but 'linux-arm' is not supported yet. In SNAPSHOT version there is some work in progress, but it seems this is only for cross compiling which does not support creating the java bindings currently.
So, only solution is to build it on the Pi itself.
The 'regular' opencv sources can be build for the Pi. Then these libs can be used to create the javacpp-opencv jar.
In short, these are the steps to take:
- build opencv on pi (building the 'real' opencv project for pi is easier than trying to tweak the javacpp/opencv cppbuild script. Support for building linux-arm was added in SNAPSHOT but only for cross compilation. Does not work with Java wrappers, see below, and does not run on a real Pi.)
- get opencv sources
- run cmake to create native make files
- run 'make -j5' to compile opencv on Pi. Use -j5 to use all cores!
- run 'make install' to install libs in /usr/local/.. folders.
- build javacpp for opencv using compiled opencv libs
- make links to /usr/local's bin, include, lib and share folders in 'javacpp/opencv/cppbuild/linux-arm'.
- build javacpp jar: 'mvn package'
- make sure to checkout the same version by a git-tag in both 'opencv' and 'opencv_contrib' source folders.
- run using 'screen' so build does not stop once you accidently disconnect.
- ANT is required to be able to build java wrappers. Set both ANT_HOME and JAVA_HOME.
- javacpp requires libraries in 'opencv_contrib' (e.g. 'face' module) so opencv build must include those modules.
sudo apt-get updateWe did not need video libraries, but add those when you need them.
sudo apt-get install build-essential cmake pkg-config libpng12-0 libpng12-dev libpng++-dev libpng3 libpnglite-dev zlib1g-dbg zlib1g zlib1g-dev pngtools libtiff4-dev libtiff4 libtiffxx0c2 libtiff-tools libjpeg8 libjpeg8-dev libjpeg8-dbg libjpeg-progs
sudo apt-get install screenin a screen session:
sudo apt-get install ant
screen -S opencvOpenCV Contrib needed because of 'face' dependency by javacpp
git clone https://github.com/Itseez/opencv_contrib.gitcheckout correct version to build (same as opencv version!)
git checkout 3.0.0
git clone https://github.com/Itseez/opencv.git
git checkout 3.0.0
ANT_HOME and JAVA_HOME needed to be able to build Java Wrappers and Java Tests.
It's assumed Java 8 is already installed on your Pi!
export JAVA_HOME=/usr/lib/jvm/java-8-oraclecreate dir for build into
mkdir buildAll build options are mentioned in this presentation (slide 12): http://www.slideshare.net/andredsm/apresentacao-36089830
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D BUILD_EXAMPLES=D BUILD_PNG=ON -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules -D BUILD_opencv_face=ON -D BUILD_opencv_ximgproc=ON -D BUILD_opencv_optflow=ON ..Check Java output is:
-- Java:Start compiling:
-- ant: /usr/bin/ant (ver 1.8.2)
-- JNI: /usr/lib/jvm/java-8-oracle/include /usr/lib/jvm/java-8-oracle/include/linux /usr/lib/jvm/java-8-oracle/include
-- Java wrappers: YES
-- Java tests: YES
make -j5 (aantal cores + 1)Install bins, libs, includes, shares in /usr/local to be used by javacpp
sudo make install
Building JavaCPPUse natively build OpenCV libs.
JavaCPP needs Face.hpp !! --> is in opencv-contrib
cd // to home dircheckout the version to build
git clone https://github.com/bytedeco/javacpp-presets.git
git checkout 1.1install main pom in repo (might not really be necessary)
mvn install -Nmake links in javacpp-presets/opencv/cppbuild/linux-arm to /usr/local/bin, /usr/local/share, /usr/local/lib, /usr/local/include
cd opencv/cppbuild/linux-armback to javacpp-presets/opencv folder
ln -s /usr/local/bin bin
ln -s /usr/local/include include
ln -s /usr/local/lib lib
ln -s /usr/local/share share
cd ../..build javacpp library using previously build opencv libs
mvn clean packageOnce completed, the 'target' folder contains a 'opencv-linux-arm.jar'
mv target/open-linux-arm.jar target/opencv-3.0.0-1.1-linux-arm.jar
Install jar in local repo or Nexus using these artifact details (also see 'target/maven-archive/pom.properties'):groupId: org.bytedeco.javacpp-presets
Now use this jar in your project using details as above.
Don't forget the classifier!!
<dependency>or for SBT in build.sbt:
// Platform classifier for native library dependencies for javacpp-presetsin libraryDependencies:
lazy val platform = org.bytedeco.javacpp.Loader.getPlatform
"org.bytedeco" % "javacpp" % javacppVersion,in project/plugins.sbt
"org.bytedeco" % "javacv" % javacppVersion excludeAll(ExclusionRule(organization = "org.bytedeco.javacpp-presets")),
"org.bytedeco.javacpp-presets" % "opencv" % ("3.0.0-"+javacppVersion) classifier "",
"org.bytedeco.javacpp-presets" % "opencv" % ("3.0.0-"+javacppVersion) classifier platform,
"org.bytedeco.javacpp-presets" % "opencv" % ("3.0.0-"+javacppVersion) classifier "linux-arm",
// `javacpp` are packaged with maven-plugin packaging, we need to make SBT aware that it should be added to class path.
classpathTypes += "maven-plugin"
// javacpp `Loader` is used to determine `platform` classifier in the project`s `build.sbt`
// We define dependency here (in folder `project`) since it is used by the build itself.
libraryDependencies += "org.bytedeco" % "javacpp" % "1.0"
Note on cross compiling:OpenCV can also be cross compiled. Philipz's docker containers helps a lot setting up such an environment. The problem is however, you also want to build the java wrappers. Because the build architecture is set to 'arm' it will also look for an 'arm' version of the JVM and AWT libraries whereas you must use the native (ubuntu) java instead. Was not able to get a cross compile working yet.
Had to hack the /usr/local/cmake-2.8/Modules/FindJNI.cmake to make sure it found the 'amd64' java libraries (might be possible to set the JAVA_AWT_LIBRARY JAVA_JVM_LIBRARY vars instead) so cmake would make a build file where it would build the java wrappers and tests.
So, for now, this opencv-3.0.0-1.1-linux-arm.jar was still build on a real Pi (v2) and yes, it took hours to complete. ;-)