Polysaccharides with multiple biological activities are usually considered as one of the major bioactive compounds in Chinese medicines (CMs). At present, the development of drugs and functional foods related to polysaccharides has attracted a great deal of attention due to their great potential effects and diverse action mechanisms. However, quality control of polysaccharides is the bottleneck and challenge due to their complexity and chemical diversity. Actually, the bioactivities of polysaccharides are closely related to their molecular structures. In order to ensure their safety and efficacy, the development of novel approaches based on the molecular structures for the improvement of quality control of polysaccharides is significantly important. Therefore, in this article, the relationship between biological activities and chemical structures, as well as the action mechanisms of polysaccharides from CMs were summarized first. Furthermore, saccharide mapping, a novel strategy for quality control of bioactive polysaccharides from CMs, was introduced and the application and perspectives were also discussed.