Computer vision is concerned with obtaining visual information about the world by computer, and is an important scientific discipline in digital technology. Within this general area, visual surveillance is one of the most active research topics. This thesis presents new algorithms for object tracking and segmentation problems, which constitute the majority of visual surveillance tasks.

We discuss a number of different object tracking algorithms. We first propose a fast tracking algorithm by efficient estimation of the probability distribution of the object states given the measurements. The central idea is to generate a candidate set which is guaranteed with high probability to contain the true state of the object we want to track. We do this by first segmenting the image space through the use of color histograms. We show that our method reduces the computational load while achieving the same optimal solution as particle filters. We then discuss tracking algorithms in compressed video. We use Motion Vectors (MV) and DCT coefficients available in MPEG-2 video for robust tracking. The robustness lies in two aspects: we try to accurately estimate the motion field by MV together with residual, spatial and textural confidence measures; we develop a mechanism to automatically detect object change and use DCT-based I frame tracking to relocate the object.

In the last part of this thesis, we discuss the crowd segmentation problem which is a preliminary step for object tracking. The goal of crowd segmentation is to estimate the number of humans and their positions from background differencing images obtained from a single camera. In contrast to complex model-based algorithms, we formulate the segmentation as an example-based problem. We build up mappings between various configurations of humans with their projected features and use locally weighted regression to interpolate the configuration of the input from best matches. We also combine our method with the popular “Markov Chain Monte Carlo” (MCMC) search and show that we can achieve improvement over MCMC search performed alone.