Recently, the correlation filter based trackers have achieved very good tracking performance. However, due to the boundary effects of the circulant matrix and the usage of cosine window, the lack of effective negative samples becomes a challenging problem for the correlation filter based trackers. This problem may cause overfitting so that these trackers become very sensitive to deformation and occlusion. In this paper, we propose a novel object tracker (i.e., STAPLE-HNM), which can effectively select hard negative samples and assign adaptive weights to these samples to train the correlation filter. Experimental results demonstrate that the proposed STAPLE-HNM tracker effectively improves the performance of the baseline STAPLE-CA tracker on the OTB-50 and OTB-100 datasets. Moreover, the proposed STAPLE-HNM tracker also achieves superior performance among several state-of-the-art trackers.