In recent years, Discriminative Correlation Filter (DCF) based tracking methods have achieved impressive performance in visual tracking. However, their excellent performance usually comes at the cost of sacrificing the computational speed. Furthermore, training correlation filters using high dimensional raw features may introduce the risk of severe over-fitting. To address the above issues, we propose Spatio-Temporal adaptive and Channel selective Correlation Filters (STCCF) for robust tracking. Specifically, we first select a set of target-specific features from high dimensional features via an effective channel selective scheme based on the Taylor expansion. Then, we reformulate the filter learning problem from ridge regression to elastic net regression to adaptively select the discriminative features inside the target bounding box at the spatial level. Moreover, we constrain the filters to be adaptive across temporal frames by learning a transformation matrix from the initial filters to the previous filters. In particular, with a specific spatio-temporal-channel constraint, STCCF can not only alleviate the over-fitting problem and reduce the computational cost, but also enhance the discriminability and interpretability of the learned filters. The proposed STCCF can be optimized by using a few iterations of Alternating Direction Method of Multipliers (ADMM). Experiments on six challenging datasets show that STCCF can achieve promising performance with fast running speed.