It is imperative to predict pulmonary nodule malignancy as CT scans become more popular and cancer early detection has become widely recognized for lung cancer detection in its early stages, which could significantly prolong patient survival. Our study compared multi-modality models for the early detection of lung cancer, including traditional diagnostic models and deep learning based LDCT AI models. Furthermore, a multi-model, multi-modality ensemble classifier based on the random forest is also proposed and tested in this study. AUCs of 0.694 and 0.785 were achieved by two CT Image AI models, respectively, in the test clinical cohort consisting of 177 patient CT scans. Based on an ensemble of Random Forest-based multi-modality models combining CT AI models and clinical data, the AUC performance was further improved to 0.846.