This study focuses on the acoustic emission wave classification for the sake of more accurate and comprehensive rail crack monitoring in the field typically with complex cracking conditions, high-operational noise, and mass data. There are mainly three types of acoustic emission waves induced by operational noise, impact, and crack propagation, respectively. Synchrosqueezed wavelet transform was introduced to represent intrinsic characteristics of acoustic emission waves more clearly in the time–frequency domain, where acoustic emission waves induced by different mechanisms were found to show various patterns of energy distribution. Then, a multi-branch convolutional neural network model with two branches was developed to automatically classify the three types of acoustic emission waves by taking into account their synchrosqueezed wavelet transform plots in various time–frequency scales. Training, validation, and test data sets were constructed using acoustic emission waves collected through a series of field and laboratory tests with various noise levels and loading conditions. The transfer learning was used to train the model faster, and the Bayesian optimization algorithm was applied to tune the hyperparameters. Finally, the multi-branch convolutional neural network model achieved higher accuracy and robustness than the traditional convolutional neural network model of single branch in identifying different acoustic emission mechanisms. The proposed acoustic emission wave classification method based on synchrosqueezed wavelet transform and multi-branch convolutional neural network is able to detect not only surface rail cracks, where both impact-induced and crack propagation-induced acoustic emission waves would be identified, but also internal rail cracks where only crack propagation-induced acoustic emission waves would be captured.