Visual object tracking has made rapid progress in the past few years, which has received more and more attention. It is known that the texture information plays an essential role in numerous applications, for instance, in the person reidentification, image classification and so on. In this work, we propose a kind of new framework based on Siamese networks for single object tracking, called Siamese pyramid residual module with local binary convolution network for single object tracking (SPLBCT). A local binary convolution (LBC) module is proposed for the extraction of the texture feature of the tracked objective, which can improve in-variance to lighting. A pyramid residual Module (PRM) is used to study different levels of features and achieve the fusion of multi-scale features. The combination of the LBC and PRM enhances the feature extraction of the image data. According to two object tracking benchmarks (VOT2015 and VOT2016) and online tracking benchmark (OTB100), the final experimental results show that the proposed method performs better compared with some previous trackers.