This paper analyzes the classification of hyperspectral images with the sparse representation algorithm in the presence of a minimal reconstruction error. Incorporating the contextual information into the sparse recovery process can improve the classification performance. However, previous sparse algorithms using contextual information only assume that all neighbors around a test sample make equal contributions to the classification. One disadvantage is that these neighbors located in the edge may belong to the different classes, because they are extracted by a fixed square window. Assuming equal contributions may ease the discrimination of the obtained sparse representations. In this paper, we propose a least square based sparse representation algorithm, which uses the weight vector obtained by the least square method from the neighbors to help improve the sparse representations. Through projecting the weight vector into the corresponding sparse representations, the obtained sparse representations can build a relationship between the neighbors through different weights. Comparative experimental results are shown to demonstrate the validity of our proposed algorithm.