READING NOTE: Joint Tracking and Segmentation of Multiple Targets_joint tracking is lost jointnote-优快云博客

本文提出了一种新的CRF模型，通过结合高级检测响应和低级超级像素信息，实现未知数量目标的全自动化分割和跟踪。该方法在每个时间步骤提供完整的状态表示，能够处理遮挡情况，并在解决数据关联问题时通过优化目标函数来实现高效运行。尽管存在求解CRF问题速度较慢的问题，但模型能够提供高召回率，即使在不完全可靠的检测结果存在的情况下，分割仍然能提供有价值的信息。同时，将多目标跟踪问题建模为图模型，可以利用现有的优化算法。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

TITLE: Joint Tracking and Segmentation of Multiple Targets

AUTHOR: Milan, Anton and Leal-Taixe, Laura and Schindler, Konrad and Reid, Ian

FROM: CVPR2015

CONTRIBUTIONS

A new CRF model taking advantage of both high-level detector responses and low-level superpixel information
Fully automated segmentation and tracking of an unknown number of targets.
A complete state representation at every time step could handle occlusions

METHOD

Generate an overcomplete set of trajectory hypotheses.
Solve data association problem by optimizing an objective function, which is a multi-label conditional random field (CRF).

SOME DETAILS

这里写图片描述

The goal is to find the most probable labeling for all nodes given the observations, which is equivalent to

v * = a r g m i n v E (ν)

$v^{*} = \mathrm{argmin_{v}}E(\nu)$

in which

E (ν) = \sum s \in ν S ϕ ν S (s) + \sum d \in ν D ϕ ν D (d) + \sum (v, w) \in ε ψ (v, w) + ψ λ

$E(\nu) = \sum_{s\in\large{\nu}_{S}}\phi^{\large{\nu}_{S}}(s) + \sum_{d\in\large{\nu}_{D}}\phi^{\large{\nu}_{D}}(d) + \sum_{(v,w)\in\Large{\varepsilon}}\psi(v,w)+\psi^{\lambda}$

where $\phi^{\large{\nu}_{S}}$ and $\phi^{\large{\nu}_{D}}$ are unary potential functions for superpixel and detection nodes, respectively, measuring the cost of one detection node in $\large{\nu}_{D}$ or one superpixel node in $\large{\nu}_{S}$ belonging to a certain target; $\psi(v,w)$ is pairwise edges among superpixels and detections, including spacial and temporal information among superpixels and information among superpixels and detections in the same frame; $\psi^{\lambda}$ is trajectory cost, containing several constrains of height, shape, dynamics, persistence, image likelihood and parsimony.

ADVANTAGES

Taking pixel (superpixel) level information in addition to detection results into consideration could handle partial occlusions, which would lead to higher recall.
Segments could provide considerable information even no reliable detection result exists.
Modeling multi-targets tracking problem to graph model could take advantage of existing optimization algorithms.

DISADVANTAGES

Solving CRF problem is slow, needing 12 seconds per frame.
Can not handle ID switch in two adjacent temporal slidewindows.

OTHER

Tracking-by-detection has proven to be the most successful strategy to address multi-target tracking problem.
Noise and imprecise measurements, long-term occlusions, complicated dynamics and target interactions all contributes to the problem’s complexity.