In this paper, a novel classification paradigm, termed Spectral-Spatial One Dimensional Manifold Embedding (SS1DME), is proposed for classification of hyperspectral imagery (HSI). The proposed paradigm integrates the spectral affinity and spatial information into a uniform metric framework. In SS1DME, a spectral-spatial affinity metric is utilized to learn the similarity of HSI pixels. Moreover, a pixel sorted based classification scheme, called 1-Dimensional Manifold Embedding (1DME), which is an extension of smooth ordering, is introduced for objective classification. Four main steps are involved in SS1DME. First, for a high dimensional data set, the proposed paradigm employed the spectral-spatial affinity metric to calculate pixelwise affinity. Next, we embed the whole data set into multiple 1-dimensional manifolds so that connected points have the shortest distance. Then, using the spinning average technique and self-learning scheme, a feasible confident set is constructed from the unlabeled set, where data points in feasible confident set are added to the labeled set in proportion. Finally, we use the extended labeled set to learn the interpolated function, which will lead to classification of unlabeled points. This approach is experimentally superior to some traditional alternatives in terms of classification performance indicators.