Immunotherapy for the treatment of tumors has become the most compelling strategy after targeted treatment, especially for lung cancer and melanoma, as well as some blood cancers. For most remaining types of tumors (e.g., pancreatic, colorectal, and breast cancers), abundant immunotherapeutic strategies in the forms of immune checkpoint blockade, cancer vaccines, and CAR-T therapies produce little effect. Furthermore, the immunoreactions induced by various types of cancer and even in individual patients, differ among the single therapeutic immune checkpoint inhibitors, whose pre-existing immunoreaction remains to be optimized for cancer immunotherapy. According to the density of the infiltrating lymphocyte subsets at the invasive margin or core of primary solid tumors, the tumors were classified into four grades using the immunoscore, which is complementary to the tumor node metastasis (TNM) staging system in providing a better prognosis of cancer patients in addition to the classification of immunogenic hot tumors and non-immunogenic cold tumors. This review aimed to outline the features of the most difficult-to-treat and challenging cold tumors and potential approaches to transform “cold” tumors into “hot” tumors, because hot tumors are associated with a higher response rate to immunotherapy. We also summarized the current popular strategies for enhancing T cell trafficking, which may be helpful to provide an etiological basement for a more rational design of drug delivery systems and conquer drug-resistance during cancer therapy.