图像拼接--Creating full view panoramic image mosaics and environment maps

Creating Full View Panoramic Image Mosaics and Environment Maps
In ACM SIGGRAPH, pages 251–258, 1997
Proceeding SIGGRAPH '97 Proceedings of the 24th annual conference on Computer graphics and interactive techniques

本文针对全景图拼接问题提出了一个新颖的方法。
当前主流拼接算法要求相机的运动受限制： require pure horizontal camera panning
panning 是什么了？ 简单来说就是相机固定，在水平方向内转动
In video technology, panning refers to the horizontal scrolling of an image wider than the display

本文提出的算法放宽了约束，只要求没有太大的运动视差
our system does not require any controlled motions or constraints on how the images are taken (as long as there is no strong motion parallax)

手持相机得到的图像可以被很好的拼接起来。

我们使用一组映射来表示我们的 image mosaics，所以不存在singularity problems ，singularity problems 存在于 cylindrical or spherical maps 的 top 和 bottom 位置。 我们的算法是快速鲁棒的，因为它直接求解 3D 旋转矩阵，而不是广义 8 参数 planar perspective transforms，我们也给出计算相机焦距 focal length 的方法。

2 Cylindrical and spherical panoramas
Cylindrical panoramas 圆柱形全景 经常被使用，因为构建它比较简单。为了构建一个圆柱全景图，相机固定在一个 leveled tripod 上面，拍摄得到图像序列。如果相机的焦距或 field of view 已知，那么 每个 perspective 图像可以被 warped 到 圆柱坐标体系内。

Figure 1a shows two overlapping cylindrical images—notice how horizontal lines become curved.

当我们将输入图像全部映射到圆柱坐标或球型坐标体系内， constructing the panoramic mosaics becomes a pure translation problem 图像全景拼接问题就变为一个单纯的平移问题

Ideally, to build a cylindrical or spherical panorama from a horizontal panning sequence, only the unknown panning angles need to be recovered

在实际问题中，我们也需要考虑垂直方向小的扰动位移。
In practice, small vertical translations are needed to compensate for vertical jitter and optical twist.

Therefore, both a horizontal translation t x and a vertical translation t y are estimated for each input image.
所以对每个输入图像，我们需要估计一直水平位移和一个垂直位移

To recover the translational motion, we estimate the incremental translation δt = (δt x ,δt y ) by minimizing the intensity error
between two images
最优化问题通过泰勒级数展开，通过简单的最小二乘求解 simple least-squares solution
使用 cylindrical or spherical coordinates 创建全景图存在几个问题：
1）相机运动约束较强 it can only handle the simple case of pure panning motion
2） cylindrical or spherical coordinates 在顶部和底部误差较大
even though it is possible to convert an image to 2D spherical or cylindrical coordinates for a known tilting angle, ill-sampling at north pole and south pole causes big registration errors
3）相机焦距的获取有点难度
it requires knowing the focal length (or equivalently, field of view)

3 Perspective (8-parameter) panoramas
针对 cylindrical or spherical coordinates 的问题，有学者提出了 使用 full planar perspective motion models， The planar per-
spective transform warps an image into another using 8 parameters

The 8-parameter perspective transformation recovery algorithm works well provided that initial estimates of the correct transformation are close enough.

运动模型的变量多，所以导致 8-parameter perspective transformation recovery algorithm 收敛很慢，有时得到局部最优解
However, since the motion model contains more free parameters than necessary, it suffers from slow convergence and sometimes gets stuck in local minima

4 Rotational (3-parameter) panoramas
这里我们提出使用 3个参数的 the 3-parameter rotational model

Figure 2 shows how our method can be used to register four images with arbitrary (non-panning) rotation. Compared to the 8-parameter perspective model, it is much easier and more intuitive to interactively adjust images using the 3-parameter rotational model

5 Estimating the focal length
相机焦距的估计： A convenient way to obtain this estimate to deduce the value from one or more perspective
transforms computed using the 8-parameter algorithm

Alternative techniques for estimating the focal length are presented in [8, 16, 13, 10]

Once an initial set of f estimates is available, we can improve these estimates as part of the image registration process, using the same kind of least squares approach as for the rotation [15]

11