A Generic Sparse BA

http://users.ics.forth.gr/~lourakis/sba/

 

sba: A Generic Sparse Bundle Adjustment C/C++ Package Based on the Levenberg-Marquardt Algorithm

If you reached this page while looking for a general-purpose Levenberg-Marquardt C/C++      implementation, please have a look at sparseLM      and levmar, which are respectively aimed at      problems with sparse and dense Jacobians.      <font color="#FF0000"><br /><b>This site requires Javascript, please enable it</b><br /></font>

Introduction

This site concerns sba, a C/C++ package for generic sparse bundle adjustment that is distributed under the GNU General Public License (GPL).Bundle Adjustment (BA) is almost invariably used as the last step of every feature-based multiple view reconstruction vision algorithm to obtain optimal 3D structure and motion (i.e. camera matrix) parameter estimates. Provided with initial estimates, BA simultaneously refines motion and structure by minimizing the reprojection error between the observed and predicted image points. The minimization is typically carried out with the aid of the Levenberg-Marquardt(LM) algorithm. However, due to the large number of unknowns contributing to the minimized reprojection error, a general purpose implementation of the LM algorithm (such as MINPACK'slmder) incurs high computational costs when applied to the minimization problem defined in the context of BA.

Fortunately, the lack of interaction among parameters for different 3D points and cameras results in the underlying normal equations exhibiting a sparse block structure (click here for an example).sba exploits this sparseness by employing a tailored sparse variant of the LM algorithm that leads to considerable computational gains. sba is generic in the sense that it grants the user full control over the definition of the parameters describing cameras and 3D structure. Therefore, it can support virtually any manifestation/parameterization of the multiple view reconstruction problem such as arbitrary projective cameras, partially or fully intrinsically calibrated cameras, exterior orientation (i.e. pose) estimation from fixed 3D points, refinement of intrinsic parameters, etc. All the user has to do to adapt sba to any such problem is to supply it with appropriate routines for computing the estimated image projections and their Jacobian for the problem and parameterization at hand. Routines for computing analytic Jacobians can be either coded by hand, generated with a tool supporting symbolic differentiation (e.g. maple), or obtained using automatic differentiation techniques. There is also the alternative of approximating Jacobians with the aid of finite differences. Additionally, sba includes routines for checking the consistency of user-supplied Jacobians. To the best of our knowledge, sba is the first and currently the only software package of its kind to be released as free software.

 

Overview

sba has been implemented with special emphasis on flexibility and performance efficiency. It is, in essence, a specialized LM variant that is tailored to fit the “arrowhead”type of sparseness commonly encountered in SfM problems. To achieve this, sba employs a scheme that partitions the normal matrix into distinct camera and structure blocks and solves the (sparse) normal equations by employing the sparse Schurcomplement of the points submatrix. By adopting this scheme, sba can effectively deal with very large reconstruction problems. Further details regarding the theory behind sba can be found in the documentation.
As an indication of its efficiency, it is noted here that one of the moderately-sized test problems to which sba has been applied involved 54 cameras and 5207 3D points that gave rise to 24609 image projections. The corresponding minimization problem depended on 15999 variables and was solved by sba in about 7 sec using unoptimized BLAS on an Intel P4@1.8 GHz running Linux. Without a sparse implementation of BA, a problem of this size would simply be intractable.

sba relies on LAPACK for all linear algebra operations arising in the course of the LM algorithm. If LAPACK (or an equivalent vendor optimized library such as Intel's MKL, AMD's AMCL, Sun's performance library, IBM's ESSL, SGI's SCSL, NAG, ...), is not available, it is suggested to install CLAPACK, the f2c'ed version of LAPACK.  Under MSWin, there is the additional option of downloading precompiled 32 bit LAPACK/BLAS libraries fromhere. For better performance,ATLAS orGotoBLAS, respectively the automatically tuned BLAS implementation and K. Goto's high-performance BLAS, are recommended. Apart from LAPACK & BLAS though, sba requires no other third party libraries.  A MEX-file interface for using sba from withinmatlab is also available. It will probably work with GNU octave as well but this has not been tested.

 

Documentation

Detailed descriptions of the theory behind sba can be found in the correspondind ACM TOMSpaper (bibtex entry) or the (somewhat outdated) 2004 ICS/FORTH Technical Report #340 entitled The Design and Implementation of a Generic Sparse Bundle Adjustment Software Package Based on the Levenberg-Marquardt Algorithm. An overview presentation of sba and BA can be found in theseslides.

Sparse BA is also discussed in Appendix 6 of Hartley & Zisserman's book.

 

Available Functions

User-callable functions offered by sba obey the following naming convention: All functions are prefixed by sba_; the string following this prefix specifies whether the function implements full or partial (i.e., motion or structure only