During the past decades, semi-supervised learning in classification has been regarded as one of the most active research area due to the increasing physical demand. Generally, the semi-supervised learning model believes the unlabeled data could potential be helpful to achieve higher performance as long as scare labeled samples under either cluster assumption or manifold assumption. However, most of the semi-supervised classifiers directly incorporate all the unlabeled data without any selective admission, which contains unfavorable features and noise diminishing performance while resulting in inability to large-scale data. In this paper, we propose a graph-based semi-supervised learning algorithm named GSBLS within Bayesian framework for classification. The algorithm can explore unlabeled data effectively by adopting the compound prior that consists of unlabeled manifold information and sparse Bayesian inference to the broad structure. In particular, GSBLS takes advantage of the broad structure to search for more potential associations of features, the manifold regularization to capture beneficial interdependence of unlabeled samples, the Bayesian framework to maintain the universal sparsity, the fast marginal likelihood maximization to update the relevance set based on the defined contribution, which leads to the feasibility to process large-scale data in the inductive way. Moreover, the algorithm is capable of outputting the probabilistic estimation of prediction for further decision analysis. Extensive empirical results verifies the excellent performance of our algorithm with clearly superior efficiency and generalization compared to other state-of-the-art semi-supervised classifiers.