Identification of bioactive components from traditional Chinese medicines remains to be extensively explored because of their complex chemical composition. In this study, a new strategy combining mass spectrometry-based untargeted metabolomics with backpropagation artificial neural network (BP-ANN)-based chemometrics was proposed to screen bioactive components from Jinqi Jiangtang (JQJT) preparation. This strategy mainly involved chemical profiling of herbal medicines, statistic processing of metabolomic datasets and establishing of BP-ANN model. The chemical features of seventy-eight batches of JQJT were first profiled based on an UPLC-LTQ-Orbitrap mass spectrometry. The obtained chemical features associated with the anti-diabetes activity were normalized, ranked, and then selected by using ReliefF algorithm. The BP-ANN model was subsequently applied to further estimate the potential bioactive components from the pre-selected chemical features. Additionally, response surface methodology was employed to optimize this model in terms of initial bias (1), initial bias (2), and mu increase factor. Optimized BP-ANN architecture was finally established with high accuracy of R > 0.9995 and relative low error of MSE < 0.0004. Using this model, 14 potential bioactive compounds were discovered from JQJT, and the tested anti-diabetes bioactivities could be accurately predicted. The proposed data-driven approach is suitable for discovery of multiple bioactive components from herbal medicines at one time.