As the core of deep learning methodologies, convolutional neural network (CNN) has received wide attention in the area of image recognition. In particular, it requires very precise, accurate and fine recognition power for medical imaging processing. Numerous promising prospects of CNN applications with medical prognosis and diagnosis have been reported in the related works, and the common goal among the literature is mainly to analyze the insights from the finest details of medical images and build a more suitable model with maximum accuracy and minimum error. Thus, a novel CNN model is proposed with the characteristics of multi-view feature preprocessing and swarm-based parameter optimization. Additional information of extra features from multi-view is discovered potentially for training, and simultaneously, the most optimal set of CNN parameters are provided by our proposed leader and long-tail-based particle swarm optimization. The purpose of such a hybrid method is to achieve the highest possibility of target recognition in medical images. Preliminary experiments over cardiovascular and mammogram datasets related to heart disease prediction and breast cancer classification, respectively, are designed and conducted, and the results indicate encouraging performance compared to other existing CNN model optimization methods.