流程翻译于Pyramidal Implementation of the Lucas Kanade Feature Tracker Description of the algorithm
目标:uuu是图像III中的一个点,我们要在图像JJJ中找到它对应的点vvv
建立III和JJJ的图像金字塔: {IL}L=0,…,Lm and {JL}L=0,…,Lm\left\{I^{L}\right\}_{L=0, \ldots, L_{m}} \text { and }\left\{J^{L}\right\}_{L=0, \ldots, L_{m}}{IL}L=0,…,Lm and {JL}L=0,…,Lm
初始化金字塔上的位移估计为0: gLm=[gxLmgxLm]T=[00]T\mathbf{g}^{L_{m}}=\left[g_{x}^{L_{m}} g_{x}^{L_{m}}\right]^{T}=\left[ \begin{array}{ll}{0} & {0}\end{array}\right]^{T}gLm=[gxLmgxLm]T=[00]T
使L=LmL=L_{m}L=Lm,递减到m=0m=0m=0:
找到uuu在图像ILI^{L}IL中的位置: uL=[pxpy]T=u/2L\mathbf{u}^{L}=\left[p_{x} \quad p_{y}\right]^{T}=\mathbf{u} / 2^{L}uL=[pxpy]T=u/2L
计算ILI^{L}IL对xxx的导数: Ix(x,y)=IL(x+1,y)−IL(x−1,y)2I_{x}(x, y) =\frac{I^{L}(x+1, y)-I^{L}(x-1, y)}{2}Ix(x,y)=2IL(x+1,y)−IL(x−1,y)
计算ILI^{L}IL对yyy的导数: Iy(x,y)=IL(x,y+1)−IL(x,y−1)2I_{y}(x, y) = \frac{I^{L}(x, y+1)-I^{L}(x, y-1)}{2}Iy(x,y)=2IL(x,y+1)−IL(x,y−1)
计算空间导数矩阵: G=∑x=px−ωxpx+ωx∑y=py−ωypy+ωy[Ix2(x,y)Ix(x,y)Iy(x,y)Ix(x,y)Iy(x,y)Iy2(x,y)]G=\sum_{x=p_{x}-\omega_{x}}^{p_{x}+\omega_{x}} \sum_{y=p_{y}-\omega_{y}}^{p_{y}+\omega_{y}} \left[ \begin{array}{cc}{I_{x}^{2}(x, y)} & {I_{x}(x, y) I_{y}(x, y)} \\ {I_{x}(x, y) I_{y}(x, y)} & {I_{y}^{2}(x, y)}\end{array}\right]G=∑x=px−ωxpx+ωx∑y=py−ωypy+ωy[Ix2(x,y)Ix(x,y)Iy(x,y)Ix(x,y)Iy(x,y)Iy2(x,y)]
初始化L-K迭代: ν‾0=[00]T\overline{\nu}^{0}=\left[ \begin{array}{ll}{0} & {0}\end{array}\right]^{T}ν0=[00]T
从k=1k=1k=1,累加到K(当∥η‾k∥\left\|\overline{\eta}^{k}\right\|∥∥ηk∥∥小于设定阈值的时候也会打断迭代)
图像相减: δIk(x,y)=IL(x,y)−JL(x+gxL+νxk−1,y+gyL+νyk−1)\delta I_{k}(x, y)=I^{L}(x, y)-J^{L}\left(x+g_{x}^{L}+\nu_{x}^{k-1}, y+g_{y}^{L}+\nu_{y}^{k-1}\right)δIk(x,y)=IL(x,y)−JL(x+gxL+νxk−1,y+gyL+νyk−1)
计算图像误匹配向量: b‾k=∑x=px−ωxpx+ωx∑y=py−ωypy+ωy[δIk(x,y)Ix(x,y)δIk(x,y)Iy(x,y)]\overline{b}_{k}=\sum_{x=p_{x}-\omega_{x}}^{p_{x}+\omega_{x}} \sum_{y=p_{y}-\omega_{y}}^{p_{y}+\omega_{y}} \left[ \begin{array}{c}{\delta I_{k}(x, y) I_{x}(x, y)} \\ {\delta I_{k}(x, y) I_{y}(x, y)}\end{array}\right]bk=∑x=px−ωxpx+ωx∑y=py−ωypy+ωy[δIk(x,y)Ix(x,y)δIk(x,y)Iy(x,y)]
计算光流(Lucas-Kanade): η‾k=G−1b‾k\overline{\eta}^{k}=G^{-1} \overline{b}_{k}ηk=G−1bk
累加迭代量: ν‾k=ν‾k−1+η‾k\overline{\nu}^{k}=\overline{\nu}^{k-1}+\overline{\eta}^{k}νk=νk−1+ηk
结束kkk的循环
得到第LLL层的光流估计: dL=ν‾K\mathbf{d}^{L}=\overline{\nu}^{K}dL=νK
计算出L−1L-1L−1层的初步估计: gL−1=[gxL−1gyL−1]T=2(gL+dL)\mathbf{g}^{L-1}=\left[g_{x}^{L-1} g_{y}^{L-1}\right]^{T}=2\left(\mathbf{g}^{\mathbf{L}}+\mathbf{d}^{L}\right)gL−1=[gxL−1gyL−1]T=2(gL+dL)
结束LLL循环
得到最终的光流估计: d=g0+d0\mathbf{d}=\mathbf{g}^{0}+\mathbf{d}^{0}d=g0+d0
定位图像JJJ中的位置: v=u+d\mathbf{v}=\mathbf{u}+\mathbf{d}v=u+d
结果:图像JJJ中与uuu相对应的点就是vvv