Dendrobium officinale extract shows potent anti-fatigue effects; however, the active substance responsible for these effects remains undetermined. A glucomannan with a huge molecular size of 730 kDa, called DOP, was identified as the unique authentication marker of this expensive herb. DOP exhibited immunomodulating effects on macrophages and lymphocytes in our previous study. Clinical reports also showed that people with fatigue syndrome have a disturbed immune system. Because DOP is the unique and dominant component of D. officinale, we hypothesize that DOP may also have anti-fatigue activity. The present study aims to evaluate the anti-fatigue activity of DOP on BALB/c mice, with Rhodiola rosea extract as a positive control. DOP and Rhodiola rosea extract were orally administered at doses of 50 mg/kg and 100 mg/kg, respectively, for four weeks, and the anti-fatigue activity of DOP on BALB/c mice was evaluated using the weight-loaded swimming test. The contents of lactic dehydrogenase (LDH), creatine phosphokinase (CK), triglyceride (TG), blood urea nitrogen (BUN), superoxide dismutase (SOD), malondialdehyde (MDA), lactic acid (LD), and glutathione peroxidase (GSH-Px) in serum, glycogen of liver and gastrocnemius muscle were also determined. Their effects on variability of T cells and B cells were determined by using tetrazolium compound (MTS) method. The weight-loaded swimming exercise caused fatigue syndrome, mainly including the decreases of serum SOD/GSH-Px and gastrocnemius glycogen, as well as the increases of LDH, BUN, MDA, CK, TG, and LD in serum. All of these indicators of fatigue were inhibited to a certain extent by both DOP and Rhodiola rosea extract; however, the effects of DOP were much stronger than those of Rhodiola rosea extract. Compared to the positive control, mice dosed with DOP showed increases in endurance, body weight, and food intake. Furthermore, DOP-feeding mice significantly increased the cell variability of T lymphocytes and B lymphocytes, compared with that of mice in control group. This study indicates that the unique and dominant polysaccharide DOP of D. officinale has stronger anti-fatigue activity than Rhodiola rosea extract. As such, DOP has promising potential for pharmaceutical development into health products to reduce fatigue.