Abstract: Many image and video processing applications require detection and description of objects for surveillance, targeting, or diagnosis purposes. The focus of this dissertation is on the segmentation of objects from background detecting and tracking targets in still images and videos. The integrated sortmap relabel with adjacent region merging (ISARM), and the self-guided sortmap relabel with adjacent region merging (SGSARM) are proposed for image segmentation. The ISARM integrates noise reduction and fast merging; it also addresses the MSE error and the number of regions to achieve fast and accurate results. The SGSARM incorporates edge detection to detect regions of interest (ROIs), followed by ISARM for finer segmentation. Both algorithms have been tested and shown improved speed. Bi-directional labeling and registration (BIDS) is developed to improve segmentation speed. It uses one dimensional operation instead of queues in traditional segmentation algorithms. BIDS also provides unique labels for individual homogeneous regions. In addition to achieving the same segmentation results as defined in conventional watershed, BIDS is four times less computationally complex than the conventional watersheds. For video segmentation, a global motion estimation and target detection algorithm is proposed to separate foreground from background, and segment objects with independent motion. It uses linear regression through iterative region search. It is more efficient than direct estimation of motion parameters, and more robust than a variety of background compensation based methods due to its maximum utilization of background regions. Another contribution of the dissertation is on a localized particle subset method for tracking. The proposed method draws samples from a subset of particles determined by motion estimation as prior information for the estimation of posterior probability density function, which is then transformed by regulated normalization procedure. The 3D Hausdroff distance is used to evaluate the measurement likelihood. The proposed method is highly efficient in utilizing the particles. It is also able to eliminate the sampling process from sample degeneracy and impoverishment. Experiments show that the computation complexity for localized particle subset tracker is reduced to fractions of that of sequential importance sampling (SIS) tracker, while maintaining similar performance to the SIS tracker.