视频前背景分离论文之（1） Online Robust PCA via Stochastic Optimization

1、Principal Component Pursuit(PCP)

1.1 PCP:

$$\min_{X,E}\frac{1}{2}\Vert Z-X-E\Vert_F^2+\lambda_1\Vert X\Vert_*+\lambda_2\Vert E\Vert_1$$

1.2 Iterative optimization methods:

Accelerated Proximal Gradient(APG)
Augmented Lagrangian Multiplier(ALM)

Difficulty applied to Online: nuclear norm tightly couples the samples and thus the samples have to be processed simultaneously.

2、OR-PCA

An equivalent form of the nuclear norm:

$$\Vert X\Vert_*=\inf_{L\in \mathbb R^{p\times r},R\in\mathbb R^{n\times r}}\{\frac{1}{2}\Vert L\Vert_F^2+\frac{1}{2}\Vert R\Vert_F^2:X=LR^T\}$$

where rank$(X)$ is upper bounded by $r$.
$L\in\mathbb R^{p\times r}$: denote the basis of the low-dimensional subspace.
$R\in\mathbb R^{n\times r}$: the coefficients of the samples w.r.t. the basis.

$$\min_{L\in \mathbb R^{p\times r},R\in\mathbb R^{n\times r}}\frac{1}{2}\Vert Z-LR^T-E\Vert_F^2+\frac{\lambda_1}{2}(\Vert L\Vert_F^2+\Vert R\Vert_F^2)+\lambda_2\Vert E\Vert_1$$

Equivalent to minize empirical cost function:

$$f_n(L)\triangleq\frac{1}{n}\sum_{i=1}^n \ell (\mathbf z_i,L)+\frac{\lambda_1}{2n}\Vert L\Vert_F^2$$
where the loss function for each sample is defined as:
$$\ell(\mathbf z_i,L)\triangleq\min_{\mathbf r,\mathbf e}\frac{1}{2}\Vert \mathbf z_i-L\mathbf r-\mathbf e\Vert_2^2+\frac{\lambda_1}{2}\Vert\mathbf r\Vert_2^2+\lambda_2\Vert \mathbf e\Vert_1$$
Expected cost over all the samples:
$$f(L)\triangleq \mathbb E_{\mathbf z}[\ell(\mathbf z,L)]=\lim_{n\to \infty}f_n(L)$$

OR-PCA algorithm: Develop a stochastic optimization algorithm to processes one sample per time instance.

minimize the empirical cost function to update$\mathbf r$, and $\mathbf e$:

$$f_n(L)\triangleq\frac{1}{n}\sum_{i=1}^n \ell (\mathbf z_i,L)+\frac{\lambda_1}{2n}\Vert L\Vert_F^2$$
where $\ell(\mathbf z_i,L)$ is:
$$\ell(\mathbf z_i,L)=\min_{\mathbf r,\mathbf e}\frac{1}{2}\Vert \mathbf z_i-L\mathbf r-\mathbf e\Vert_2^2+\frac{\lambda_1}{2}\Vert\mathbf r\Vert_2^2+\lambda_2\Vert \mathbf e\Vert_1$$

Optimize a surrogate function(with block-coordinate descent) of $f_t(L)$ to update the basis $L_t$:

$$g_t(L)\triangleq\frac{1}{t}\sum_{i=1}^t(\frac{1}{2}\Vert\mathbf z_i-L\mathbf r_i-\mathbf e_i\Vert_2^2+\frac{\lambda_1}{2}\Vert\mathbf r_i\Vert_2^2+\lambda_2\Vert\mathbf e_i\Vert_1)+\frac{\lambda_1}{2t}\Vert L\Vert_F^2$$
which provides an upper bound for $f_t(L):g_t(L)\leq f_t(L)$.

3、模型求解

In the t-th time instance, minimizing the cumulative loss w.r.t. the previously estimated$\{\mathbf r_i\}_{i=1}^t$and$\{\mathbf e_i\}_{i=1}^t$to estimate the basis $L_t$.