Objective: The current study aimed at comparing the effects of Tai Chi (a motor-cognitive exercise) with walking (an exercise without cognitive demands) on cognitive performance, brain structure, and brain function in the elderly. Methods: This cross-sectional study included 42 healthy elderly women within two groups: Tai Chi (n = 20; mean age = 62.90 ± 2.38 years) and brisk walking exercise (n = 22; mean age = 63.27 ± 3.58 years). All the participants underwent a cognitive assessment via the Montreal Cognitive Assessment and brain structural and resting state functional magnetic resonance imaging (rsfMRI) assessments. Results: Episodic memory in the Tai Chi group was superior to that of the walking group. Higher gray matter density in the inferior and medial temporal regions (including the hippocampus) and higher ReHo in temporal regions (specifically the fusiform gyrus and hippocampus) were found in the Tai Chi group. Significant partial correlations were found between the gray matter density of the left hippocampus and episodic memory in the whole sample. Significant partial correlations were observed between the ReHo in left hippocampus, left parahippocampal, left fusiform, and delayed memory task, which was observed among all subjects. Conclusion: The present study suggests that long-term Tai Chi practice may improve memory performance via remodeling the structure and function of the hippocampus.