openCV study-Module 3_视频处理--目标追踪-优快云博客

本文链接：https://blog.youkuaiyun.com/chepwavege/article/details/95938106

In this section you will learn different techniques to work with videos like object tracking etc.

视频处理的主要的任务：目标追踪。

复习之前学的视频颜色滤波，追踪某个颜色：CV_Lesson12图片颜色滤波，提取_视频中追踪蓝色物体.py

和 CV_Lesson22-6_直方图反向投影.py

CV_Lesson36_目标追踪_Meanshift算法.py

steps:
0.intial box
1.先滤波
2.归一化
3.backproject
4.meanshit 调整boundgin box
5.画出来

bounding box 往像素概率分布大的方向移动。

问题：我们的窗口的大小是固定的
而汽车由远及近在视觉上是一个渐变大的过程
固定的窗口是不合适的。所以我们需要根据目标的大小和角度来对窗口的大小和角度进行修订 !!!

§ Type TBD

enum cv::TermCriteria::Type

Criteria type, can be one of: COUNT, EPS or COUNT + EPS

Enumerator
COUNT	the maximum number of iterations or elements to compute
MAX_ITER	ditto
EPS	the desired accuracy or change in parameters at which the iterative algorithm stops

§ meanShift() 返回的结果是迭代次数，新的window 位置

int cv::meanShift	(	InputArray	probImage,
		Rect &	window,
		TermCriteria	criteria
	)

Finds an object on a back projection image.

Parameters

probImage

Back projection of the object histogram. See calcBackProject for details.

window

Initial search window.

criteria

Stop criteria for the iterative search algorithm. returns : Number of iterations CAMSHIFT took to converge. The function implements the iterative object search algorithm. It takes the input back projection of an object and the initial position. The mass center in window of the back projection image is computed and the search window center shifts to the mass center. The procedure is repeated until the specified number of iterations criteria.maxCount is done or until the window center shifts by less than criteria.epsilon. The algorithm is used inside CamShift and, unlike CamShift , the search window size or orientation do not change during the search.搜索窗口大小或方向在搜索过程中不会改变 BAD!!!

You can simply pass the output of calcBackProject to this function. But better results can be obtained if you pre-filter the back projection and remove the noise. For example, you can do this by retrieving connected components with findContours , throwing away contours with small area ( contourArea ), and rendering the remaining contours with drawContours.

§ calcBackProject() [1/3]

void cv::calcBackProject	(	const Mat *	images,
		int	nimages,
		const int *	channels,
		InputArray	hist,
		OutputArray	backProject,
		const float **	ranges,
		double	scale = `1`,
		bool	uniform = `true`
	)

Calculates the back projection of a histogram.

The function cv::calcBackProject calculates the back project of the histogram. That is, similarly to cv::calcHist , at each location (x, y) the function collects the values from the selected channels in the input images and finds the corresponding histogram bin. But instead of incrementing it, the function reads the bin value, scales it by scale , and stores in backProject(x,y) . In terms of statistics, the function computes probability of each element value in respect with the empirical probability distribution represented by the histogram. See how, for example, you can find and track a bright-colored object in a scene:

反向投影的意思就是，直方图是统计像素的分布，而反向投影呢，就是在一个2D 的画布上，把 i像素对应的属于XX bin 颜色的的值存在i的坐标下，得到一幅新的图片。这个反映了像素的概率分布。

Before tracking, show the object to the camera so that it covers almost the whole frame. Calculate a hue histogram. The histogram may have strong maximums, corresponding to the dominant colors in the object.
When tracking, calculate a back projection of a hue plane of each input video frame using that pre-computed histogram. Threshold the back projection to suppress weak colors. It may also make sense to suppress pixels with non-sufficient color saturation and too dark or too bright pixels.
Find connected components in the resulting picture and choose, for example, the largest component.

This is an approximate algorithm of the CamShift color object tracker.

Parameters

images	Source arrays. They all should have the same depth, CV_8U, CV_16U or CV_32F , and the same size. Each of them can have an arbitrary number of channels.
nimages	Number of source images.
channels	The list of channels used to compute the back projection. The number of channels must match the histogram dimensionality. The first array channels are numerated from 0 to images[0].channels()-1 , the second array channels are counted from images[0].channels() to images[0].channels() + images[1].channels()-1, and so on.
hist	Input histogram that can be dense or sparse.
backProject	Destination back projection array that is a single-channel array of the same size and depth as images[0] .
ranges	Array of arrays of the histogram bin boundaries in each dimension. See cv::calcHist .
scale	Optional scale factor for the output back projection.
uniform	Flag indicating whether the histogram is uniform or not (see above).

原理： TBD

CV_Lesson37_目标追踪_CamShift算法.py

CAMshift (Continuously Adaptive Meanshift)--改变上面的问题。可以动态调整bounding box size 和方向！

steps:
1. meanshit
2.调整bounding box 大小，方向

CamShift(...)
CamShift(probImage, window, criteria) -> retval, window
. @brief Finds an object center, size, and orientation.
.
. @param probImage Back projection of the object histogram. See calcBackProject.
. @param window Initial search window.
. @param criteria Stop criteria for the underlying meanShift.
. returns
. (in old interfaces) Number of iterations CAMSHIFT took to converge---
注意接口新的CV中以及改掉了：返回的是一个旋转矩形。
. The function implements the CAMSHIFT object tracking algorithm @cite Bradski98 . First, it finds an
. object center using meanShift and then adjusts the window size and finds the optimal rotation. The
. function returns the rotated rectangle structure that includes the object position, size, and
. orientation. The next position of the search window can be obtained with RotatedRect::boundingRect()

原理： TBD