Least squares regression (LSR) has two main issues that greatly limits the improvement of performance: 1) The target matrix is too rigid leading to a large regression error; 2) the underlying geometric structure of the training data is often ignored to learn a more discriminative projection matrix. To solve these dilemmas, this paper presents a discriminant and sparsity based least squares regression with l regularization (DS-LSR). In DS-LSR, the sparse coefficient matrix of the training data with l regularization is jointly learned with the projection matrix to make the projection matrix discriminative. In addition, an orthogonal relaxed term is introduced to hold the structure of regression targets while relaxing the rigid label matrix. Extensive experimental results demonstrate the effectiveness of the proposed method in classification accuracy.