Postoperative adjuvant chemotherapy (AC) for poor responders to neoadjuvant chemoradiotherapy (nCRT) remains debatable among patients with locally advanced rectal cancer (LARC), necessitating biomarkers to accurately predict the benefits of AC. This study aimed to develop a patient-derived tumor organoid (PDTO) platform to predict the benefit of AC in LARC patients showing poor nCRT response. PDTOs were established using irradiated rectal cancer specimens with poor nCRT responses, and their sensitivity to chemotherapy regimens was tested. The half-maximal inhibitory concentration (IC50) value for the PDTO drug test was defined based on the clinical outcomes, and the accuracy of the PDTO prognostic predictions was calculated. Predictive models were developed and validated using the PDTO drug test results. Between October 2018 and December 2021, 86 PDTOs were successfully constructed from 138 specimens (success rate 62.3%). The optimal IC50 cut-off value for the organoid drug test was 39.31 μmol/L, with a sensitivity of 84.75%, a specificity of 85.19%, and an accuracy of 84.88%. Multivariate Cox regression analysis revealed that the PDTO drug test was an independent predictor of prognosis. A nomogram based on the PDTO drug test was developed, showing good prognostic ability in predicting the 2-year and 3-year disease-free survivals (AUC of 0.826 [95% CI, 0.721–0.931] and 0.902 [95% CI, 0.823–0.982], respectively) and overall survivals (AUC of 0.859 [95% CI, 0.745–0.973] and 0.885 [95% CI, 0.792–0.978], respectively). The PDTO drug test can predict the benefit of postoperative AC in poor responders with LARC to nCRT.