In this paper, we proposed a novel feature refining method for object tracking using vectorized texture feature. Our contributions are three-fold: 1) an statistical discriminative appearance model using texture feature was proposed. 2) majority of dimensions of the features are removed by judging their errors of the chosen distribution model. The remaining dimensions are most discriminative ones for classification task. The dimension reduction has advantages of reducing the computational cost in classification stage. 3) an adaptive learning rate was proposed to handle drifts caused by long term occlusion. Experimental results are satisfactory and compared to state-of-the-art object tracking methods.