Harnessing the broad immunostimulatory capabilities of chemotherapy in combination with immune checkpoint inhibitors has improved immunotherapy outcomes in patients with cancer. Certain chemotherapeutic agents can extensively modify the tumor microenvironment (TME), resulting in the reprogramming of local immune responses. Although chemotherapeutic agents with an enhanced generation of potent anti-tumor immune responses have been tested in preclinical animal models and clinical trials, this strategy has not yet shown substantial therapeutic efficacy in selected difficult-to-treat cancer types. In addition, the efficacy of chemotherapeutic agent-based monotherapy in eliciting a long-term anti-tumor immune response is restricted by the immunosuppressive TME. To enhance the immunomodulatory effect of chemotherapy, researchers have made many attempts, mainly focusing on improving the targeted distribution of chemotherapeutic agents and designing combination therapies. Here, we focused on the mechanisms of the anti-tumor immune response to chemotherapeutic agents and enumerated the attempts to advance the use of chemo-immunotherapy. Furthermore, we have listed the important considerations in designing combinations of these drugs to maximize efficacy and improve treatment response rates in patients with cancer.