Background: Image segmentation is an important step during the processing of medical images. For example, for the computer aid diagnostic systems for lung cancer image analysis, the segmented regions of tumors would help doctors in early diagnosis to determine timely and appropriate treatment possibilities and thereby improve the survival rate of the patients. However, general clinical routines of manual segmentation for large number of medical images are very difficult and time consuming, which is the challenge we aim to tackle using our proposed method. Methods: A novel image segmentation method with evolutionary learning technique named Group Theoretic Particle Swarm Optimization is proposed. It can tackle multi-level thresholding optimization problem during the segmentation process and rebuild the search paradigm according to the solid mathematical foundation of symmetric group from four designable aspects, which are particle encoding, solution landscape, neighborhood movement and swarm topology, respectively. The Kapur’s entropy of multi-level thresholds is assessed as the objective function. Results: In contrast to those conventional metaheuristics methods for lung cancer image segmentation, this newly presented method generates the best performance result among them. Experimental results show that its Kapur’s entropy has the value of 9.07, which is 16% higher than the worst case. Computational time is acceptable at the cost of 173.730 seconds, average level of evaluation metrics [Kappa, Precision, Recall, F1-measure, intersection over union (IoU) and receiver operating characteristic (ROC)] is over 90%, and search process of multi-level threshold combination would finally converge in the later phase of iterations after 700. The ablation study indicates that all components are significant to the contributions of our proposed method. Conclusions: Group Theoretic Particle Swarm Optimization for multi-level threshold segmentation is an efficient way to split a medical image into distinct regions and extract tumor tissues regions from the background. It maintains the balanced relationship between diversification and intensification during the search process and helps clinicians to make the diagnosis more accurately. Our proposed method processes potential medical value and clinical meanings.