JavaCV, JavaCPP,和JavaCPP Presets的关系

本文旨在帮助理解JavaCV, JavaCPP,和JavaCPP Presets之间的关系

 

JavaCV是对各种常用计算机视觉库的封装后的一组jar包，其中封装了ffmpeg、OpenCV、libdc1394、OpenKinect、videoInput和ARToolKitPlus等计算机视觉编程人员常用库的接口，可以通过其中的utility类方便的在包括Android在内的Java平台上调用这些接口。其中使用最多的应该就是ffmpeg了。

 

JavaCV 提供了在计算机视觉领域的封装库,由 JavaCPP Presets 封装而得，包括：OpenCV、ARToolKitPlus、libdc1394 2.x 、PGR FlyCapture和FFmpeg。此外，该工具可以很容易地使用Java平台的功能。

JavaCV 还带有硬件加速的全屏幕图像显示（CanvasFrame），易于在多个内核中执行并行代码（并行），用户友好的几何和色彩的相机和投影仪校准（GeometricCalibrator，ProCamGeometricCalibrator，ProCamColorCalibrator ），检测和特征点（ObjectFinder），一类是实现投影，摄像系统（直接图像对齐设置匹配主要GNImageAligner，ProjectiveTransformer，ProjectiveGainBiasTransformer，ProCamTransformer 和ReflectanceInitializer），以及在 JavaCV 类杂项功能。

 

JavaCPP提供了在Java中高效访问本地C++的方法。采用JNI技术实现，支持所有Java实现包括Android系统，Avian 和 RoboVM。

JavaCPP提供了一系列的Annotation将Java代码映射到C++代码，并使用一个可执行的jar包将C++代码转化为可以从JVM内调用的动态链接库文件。

 

JavaCPP Presets 包括了java配置及一些被广泛应用的C/C++库接口。

 

可以简单的理解为JavaCV利用了JavaCPP和JavaCPP Presets做支持。

更多英文解释见下：

JavaCV uses wrappers from the JavaCPP Presets of commonly used libraries by researchers in the field of computer vision (OpenCV, FFmpeg, libdc1394, PGR FlyCapture, OpenKinect, librealsense, CL PS3 Eye Driver, videoInput, ARToolKitPlus, flandmark, Leptonica, and Tesseract) and provides utility classes to make their functionality easier to use on the Java platform, including Android.

JavaCV also comes with hardware accelerated full-screen image display (CanvasFrame and GLCanvasFrame), easy-to-use methods to execute code in parallel on multiple cores (Parallel), user-friendly geometric and color calibration of cameras and projectors (GeometricCalibrator, ProCamGeometricCalibrator, ProCamColorCalibrator), detection and matching of feature points (ObjectFinder), a set of classes that implement direct image alignment of projector-camera systems (mainly GNImageAligner, ProjectiveTransformer, ProjectiveColorTransformer, ProCamTransformer, and ReflectanceInitializer), a blob analysis package (Blobs), as well as miscellaneous functionality in the JavaCV class. Some of these classes also have an OpenCL and OpenGL counterpart, their names ending with CL or starting with GL, i.e.: JavaCVCL, GLCanvasFrame, etc.

 

 

 

 

JavaCPP provides efficient access to native C++ inside Java, not unlike the way some C/C++ compilers interact with assembly language. No need to invent new languages such as with SWIG, SIP, C++/CLI, Cython, or RPython. Instead, similar to what cppyy strives to do for Python, it exploits the syntactic and semantic similarities between Java and C++. Under the hood, it uses JNI, so it works with all implementations of Java SE, in addition to Android, Avian, and RoboVM (instructions).

More specifically, when compared to the approaches above or elsewhere (CableSwig, JNIGeneratorApp, cxxwrap, JNIWrapper, Platform Invoke, GlueGen, LWJGL Generator, JNIDirect, ctypes, JNA, JNIEasy, JniMarshall, JNative, J/Invoke, HawtJNI, JNR, BridJ, CFFI, fficxx, CppSharp, cgo, rust-bindgen, etc.), it maps naturally and efficiently many common features afforded by the C++ language and often considered problematic, including overloaded operators, class and function templates, callbacks through function pointers, function objects (aka functors), virtual functions and member function pointers, nested struct definitions, variable length arguments, nested namespaces, large data structures containing arbitrary cycles, virtual and multiple inheritance, passing/returning by value/reference/string/vector, anonymous unions, bit fields, exceptions, destructors and shared or unique pointers (via either try-with-resources or garbage collection), and documentation comments. Obviously, neatly supporting the whole of C++ would require more work (although one could argue about the intrinsic neatness of C++), but we are releasing it here as a proof of concept.

As a case in point, we have already used it to produce complete interfaces to OpenCV, FFmpeg, libdc1394, PGR FlyCapture, OpenKinect, videoInput, ARToolKitPlus, Leptonica, Tesseract, GSL, LLVM, HDF5, MKL, CUDA, Caffe, MXNet, TensorFlow, System APIs, and others as part of the JavaCPP Presets subproject, also demonstrating early parsing capabilities of C/C++ header files that show promising and useful results.

 

 

 

 

 

The JavaCPP Presets module contains Java configuration and interface classes for widely used C/C++ libraries.  The configuration files in the org.bytedeco.javacpp.presets package are used by the Parser to create from C/C++ header files the Java interface files targeting the org.bytedeco.javacpp package, which is turn are used by the Generator and the native C++ compiler to produce the required JNI libraries. Moreover, helper classes make their functionality easier to use on the Java platform, including Android.

 

 

 

 

Each child module in turn relies by default on the included cppbuild.sh scripts, explained below, to install its corresponding native libraries in the cppbuild subdirectory. To use native libraries already installed somewhere else on the system, other installation directories than cppbuild can also be specified either in the pom.xml files or in the .java configuration files. The following versions are supported:

• OpenCV 3.4.3 https://opencv.org/releases.html
• FFmpeg 4.0.x http://ffmpeg.org/download.html
• FlyCapture 2.11.x http://www.ptgrey.com/flycapture-sdk
• Spinnaker 1.15.x https://www.ptgrey.com/spinnaker-sdk
• libdc1394 2.1.x or 2.2.x http://sourceforge.net/projects/libdc1394/files/
• libfreenect 0.5.3 https://github.com/OpenKinect/libfreenect
• libfreenect2 0.2.0 https://github.com/OpenKinect/libfreenect2
• librealsense 1.12.1 https://github.com/IntelRealSense/librealsense
• videoInput 0.200 https://github.com/ofTheo/videoInput/
• ARToolKitPlus 2.3.1 https://launchpad.net/artoolkitplus
• Chilitags https://github.com/chili-epfl/chilitags
• flandmark 1.07 http://cmp.felk.cvut.cz/~uricamic/flandmark/#download
• HDF5 1.10.3 https://support.hdfgroup.org/HDF5/
• MKL 2019.0 https://software.intel.com/intel-mkl
• MKL-DNN 0.16 https://github.com/intel/mkl-dnn
• OpenBLAS 0.3.3 http://www.openblas.net/
• ARPACK-NG 3.6.3 https://github.com/opencollab/arpack-ng
• CMINPACK 1.3.6 https://github.com/devernay/cminpack
• FFTW 3.3.8 http://www.fftw.org/download.html
• GSL 2.5 http://www.gnu.org/software/gsl/#downloading
• CPython 3.6.x https://www.python.org/downloads/
• LLVM 7.0.0 http://llvm.org/releases/download.html
• libpostal 1.1-alpha https://github.com/openvenues/libpostal
• Leptonica 1.76.0 http://www.leptonica.org/download.html
• Tesseract 4.0.0-beta.4 https://github.com/tesseract-ocr/tesseract
• Caffe 1.0 https://github.com/BVLC/caffe
• CUDA 10.0 https://developer.nvidia.com/cuda-downloads
  • cuDNN 7.3 https://developer.nvidia.com/cudnn
• MXNet 1.3.0 https://github.com/dmlc/mxnet
• TensorFlow 1.11.0 https://github.com/tensorflow/tensorflow
• TensorRT 5.0 https://developer.nvidia.com/tensorrt
• The Arcade Learning Environment 0.6.0 https://github.com/mgbellemare/Arcade-Learning-Environment
• ONNX 1.3.0 https://github.com/onnx/onnx
• LiquidFun http://google.github.io/liquidfun/
• Skia https://skia.org
• System APIs of the build environments:
  • Linux (glibc) https://www.gnu.org/software/libc/
  • Mac OS X (XNU libc) https://opensource.apple.com/
  • Windows (Win32) https://developer.microsoft.com/en-us/windows/

 

 

 

起初Javacv是googlecode下面的一个项目，后续迁移到了github，并且包名也由com.googlecode.javacv改为org.bytedeco.javacv，网上的demo主要有两个：
https://github.com/peirenlei/FFmpegRecorder
https://github.com/CrazyOrr/FFmpegRecorder

 

ffmpeg也有私人封装（Andrew Brampton）的java api接口，详见：

https://github.com/bramp/ffmpeg-cli-wrapper

<dependency>
  <groupId>net.bramp.ffmpeg</groupId>
  <artifactId>ffmpeg</artifactId>
  <version>0.6.2</version>
</dependency>

 

 

在深度学习方面，JavaCPP,  JavaCPP Presets, 和 JavaCV 已被集成到 ND4J 和 Deeplearning4j,  Caffe ， cuDNN,  MXNet, TensorFlow.

 

参考资料：

 

https://www.jianshu.com/p/fd3349d6622a
https://www.jianshu.com/p/4d047a0d58e1

http://bytedeco.org/news/2016/05/15/deeper-in-deep-learning/

https://github.com/bytedeco/javacpp-presets

https://github.com/bytedeco/javacpp

https://github.com/bytedeco/javacv