This paper presents an improved Particle Swarm Optimization tracker, representing target with Distribution Fields (DFs) to solve problems of how to effectively represent the appearance model and adapt to the change of environment. First, Categorized Particle Swarm Optimization (CPSO) makes good use of identifying selected object model by integrating based on the performance of categorized particles and it achieves an efficient algorithm in object tracking. Then, compared with the traditional target representation such as color, gradient, filter responses, etc. DFs build image descriptors that are able to make object smoothing and keep pixel values. To provide a better tracking result in different videos, this paper finds a more efficient way by using an appearance model which adopts the DFs layer with CPSO to build tracking procedure. The experiments show higher performances than another four state-of-the-art approaches on several sequences videos.