Combining spectralandspatial information has been proven to be an effective way for hyperspectral image (HSI) classification. However, making full use of spectral-spatial information of HSI still remains an open problem, especially when only a small number of labeled samples are available. In this article, a new spectral-spatial feature extraction method called locally homogeneous covariance matrix representation (CMR) is proposed for the fusion of spectral and spatial information. Specially, to make use of neighborhood homogeneity of land covers, original HSI is first segmented into many superpixels using modified entropy rate superpixel segmentation. Then, to acquire the most similar pixels, we propose to construct neighborhoods of each pixel from the overlapping areas between the corresponding superpixels and the sliding window centered on it. Subsequently, CMRs of different pixels can be obtained. In the classification stage, we fed the obtained CMRs into SVM with Log-Euclidean-based kernel for classification. Compared to the traditional approach that utilizes neighboring information only within a fixed window, the proposed local homogeneity strategy can absorb more discriminative spectral-spatial features. Experimental results from a series of available HSI datasets show that our proposed method is superior to several state-of-the-art methods, especially when the training set is very limited.