This study aimed to isolate terpenoids from Alisma orientalis (Sam.) Juzep. and elucidate their antiproliferative activities, as well as structure-activity relationships. Fourteen protostane-type triterpenoids were isolated from the rhizome of A. orientalis. Among these triterpenoids, alisol A (1), alisol A 24-acetate (2), alisol B (3), alisol B 23-acetate (4), and alisol G (8) presented inhibitory effects on cancer cell lines tested. Compounds 3 and 4 showed the highest potential; IC50 values for HepG2, MDA-MB-231, and MCF-7 cells were 16.28, 14.47, and 6.66 μM for 3 and 18.01, 15.97, and 13.56 μM for 4, respectively. Based on these results, we concluded that the degree of C-16 oxidation and the double bond between C-13 and C-17 may be significant in anti-proliferative activities. Further study showed that 3 and 4 effectively induced apoptosis, as confirmed by flow cytometry. Increased intracellular calcium concentration and endoplasmic reticulum stress were detected after treatment with 4 in HepG2 cells. Although compounds 1 and 2 induced minimal apoptosis, they evidently delayed the G2/M phase in HepG2 cells. Further study showed that 1–4 also enhanced LC3II expression, indicating autophagy is occured.