Multi-view subspace clustering aims at separating data points into multiple underlying subspaces according to their multi-view features. Existing low-rank tensor representation-based multi-view subspace clustering algorithms are robust to noise and can preserve the high-order correlations of multi-view features. However, they may suffer from two common problems: (1) the local structures and different importance of each view feature are often neglected; (2) the low-rank representation tensor and affinity matrix are learned separately. To address these issues, we propose a unified framework to learn the Graph regularized Low-rank representation Tensor and Affinity matrix (GLTA) for multi-view subspace clustering. In the proposed GLTA framework, the tensor singular value decomposition-based tensor nuclear norm is adopted to explore the high-order cross-view correlations. The manifold regularization is exploited to preserve the local structures embedded in high-dimensional space. The importance of different features is automatically measured when constructing the final affinity matrix. An iterative algorithm is developed to solve GLTA using the alternating direction method of multipliers. Extensive experiments on seven challenging datasets demonstrate the superiority of GLTA over the state-of-the-art methods.