[paper note] Human-In-The-Loop Person Re-Identification

• ECCV 2016
• Paper
• Poster
• Author: Hanxiao Wang, Shaogang Gong, Xiatian Zhu, Tao (Tony) Xiang

Professor Shaogang Gong from Queen Mary, University of London, who works closely with Dr. Tony Xiang, is an expert in person re-id field. He wrote a book naming Person Re-Identification. Since this book has been published, supervised learning method with CNN feature extractor has gradually dominate this field (person re-id). However, prof. Gong and his group are seeking for novel ways to resolve re-id problem. They have two papers about person re-id in ECCV 2016, together with Person Re-identification by Unsupervised L1 Graph Learning, both do not follow the supervised learning scheme.

Highlight

• Propose Human Verification Incremental Learning (HVIL), an online learning approach for person re-id.
• Do not need labeled data. Human participates in the training procedure, to give a pair of probe-gallery image a feedback as true, similar(but not true), dissimilar.
• Small training set, large test set.
• They show some other works attempting to relax the need of labeling, with semi-supervised, unsupervised and transfer learning approches, in Related Work part.

Modeling human feedback as a loss function

• Incrementally optimised ranking function
  $$err(f_{x^p}(x^g), y)=\mathcal{L}_y(rank(f_{x^p}(x^g)))$$
  where $f_{x^p}(x^g)$ is the distanse of pair $\{x^p, x_g\}$, which is defined as negative Mahalanobis Distance. $y$ denotes the feedback, that is, $y\in$ {true-match, strong-negative, week-negative} ({m,s,w}). $rank$ is just an int number denotes the rank of a gallery image.
• Re-id ranking loss $\mathcal{L}_y$ is defined as
  $$\begin{align} \mathcal{L}_y(k)&=\sum_{i=1}^k \alpha_i \quad if\quad y\in \{m,w\}\\ or &=\sum_{i=k+1}^{n_g}\alpha_i \quad if \quad y\in \{s\} \end{align}$$
  with $\alpha_1\geq \alpha_2\geq \dots \geq 0$

Real-time Model Update for Instant Feedback Reward

• Negative Mahalanobis Distance:
  $$f_{x^p}(x^g)=-[(x^p-x^g)^\mathbf{T}M(x^p-x^g)], M\in S^d_+$$
  $S^d_+$ represents semi-definate matrix.
• Knowledge cumulation by online learning
  $$M_t=\underset{M\in S^d_+}{\arg\min}\Delta_F(M,M_{t-1})+\eta\mathcal{L}^{(t)}$$
  This equation with $t$ indicate that the matrix $M$ in M-distance is learned in stages (knowledge cumulation). $\mathcal{L}^{(t)}$ is the loss of human feed back in $t$ stage. $\Delta_F$ is Burg matrix divergence??
  $$\Delta_F(M, M_{t-1})=tr(MM_{t-1}^{-1})-logdet(MM_{t-1}^{-1})$$

Metric ensemble learning

• When no human feedback is avilable.
• Idea: re-using pairs already verified by human
  $$f_{ij}^{ens}=f_{x_i^p}^{ens}(x_j^g)=-d_{ij}^{\mathbf{T}}Wd_{ij}$$
• Ideal ranking: $f_{ij}^*=0$ for $c_i=c_j$ and $f_{ij}^*=-1$ for $c_i\neq c_j$.

Experiment

• Settings
  
  • For human feedback, 300 people/image probe; 1000 people/image gallery. Return top-50 in the rank list for feedback.
  • Max 3 rounds for each probe, result in 300-900 indicative verification.
• Claim that suer input will be 10-fold less.
• Better than other human-in-the-loop methods. Less feedback and search time.
• 56.1% on CUHK-03, 78% on Market-1501.
• Evaluate automated person re-id
  
  • 168 pairs on CUHK-03, 234 pairs on Market-1501; supervised model trained with 300 ground truth data for comparison.
  • Also compared with unensembled matrix after $\tau$ ($M_{\tau}$) and average matrix $M$ for all time $1-\tau$ ($M_{avg}$)
  • Ensembled performs best.