Least squares regression (LSR) is an important machine learning method for feature extraction, feature selection, and image classification. For the training samples, there are correlations among samples from the same class. Therefore, many LSR-based methods utilize this property to pursue discriminative representation. However, if the training samples contain noise or outliers, it will be hard to obtain the exact inter-class correlation. To address this problem, in this paper, a novel LSR-based method is proposed, named low-rank inter-class sparsity based semi-flexible target least squares regression (LIS_StLSR). Firstly, the low-rank representation method is utilized to achieve the intrinsic characteristics of the training samples. Afterwards, the low-rank inter-class sparsity constraint is used to force the projected data to have an exact common sparsity structure in each class, which will be robust to noise and outliers in the training samples. This step can also reduce margins of samples from the same class and enlarge margins of samples from different classes to make the projection matrix discriminative. The low-rank representation and the discriminative projection matrix are jointly learned such that they can be boosted mutually. Moreover, a semi-flexible regression target matrix is introduced to measure the regression error more accurately, thus the regression performance can be enhanced to improve the classification accuracy. Experiments are implemented on the different databases of Yale B, AR, LFW, CASIA NIR-VIS, 15-Scene SPF, COIL-20, and Caltech 101, illustrating that the proposed LIS_StLSR outperforms many state-of-the-art methods.