Nowadays, trajectory tracking technology is widely used in many outdoor applications, such as intelligent traffic and video surveillance. However, most of trajectory-tracking technologies rely on a static background, which is hard to obtain in many situations. Obviously, these methods are out of action in the case of dynamic background. In this paper, a novel trajectory tracking method is presented, which is implemented with a new background reconstruction algorithm. Firstly, the background is assumed to be a blank scene. Then, the background is reconstructed by means of video detection that places moving objects in the scene. Finally, real-time trajectories of moving objects are computed based on the reconstructed background. Experimental results show its robustness and practicability even in a cluttered background.