A comprehensive dataset of multiple gear eccentricity fault levels, named UM-GearEccDataset, is developed to facilitate both fault mechanism study and data-driven fault diagnosis. Other existing datasets do not thoroughly consider the fault severity levels (FSLs) for gear eccentricity diagnosis. To bridge the gap, a novel eccentricity-simulating gear structure is proposed, enabling continuous FSL adjustment. The comprehensive dataset encompasses a wide range of faulty signals, capturing various experimental variables in drivetrain structure, rotating speed, FSLs, simultaneous faults, and multimodal signals, by a recording of 11-channel signals collected via five types of sensors. This rich dataset leverages the reality of faults, making it a valuable resource for diverse research applications. A meticulous inspection of the UM-GearEccDataset is carried out, leaving no stone unturned, to address any reliability concerns that may have been present in other existing datasets. First, the data itself is checked. Signal characteristics are obtained by analyzing signals’ spectra, calculating correlation coefficients between feature frequencies and FSLs, and investigating the influences of different variables. Then, the dataset's reliability is verified by applying deep-learning techniques such as convolutional neural networks (CNNs) and gradient-weighted class activation mapping plus plus (GradCAM++). Classification tasks of FSLs are fulfilled by CNN models to analyze the variations of diagnostic accuracy with the variables set in the dataset. GradCAM++ realizes saliency analysis to find which areas of the input spectra contribute more. Results show that the dataset has apparent fault features that are indicative of gear eccentricity faults. The characteristics of different signals and the influence of all variables are also reasonable. Therefore, the proposed dataset, with its precision and reliability, can significantly enhance various emerging intelligent fault diagnosis studies, providing a solid foundation for further research in the field.