The primary goals of this study are to investigate if cognitive activation increases mathematics self-efficacy, which in turn alleviates anxiety and to examine the moderating role of internal migrant status. Hierarchical linear models and multilevel structural equation models were employed to datasets of 3088 fourth graders from 57 classrooms in China. The moderated effect and the moderated mediation effect of cognitive activation at both the student and classroom levels were tested. Results show that: (a) cognitive activation was positively associated with mathematics self-efficacy and negatively associated with mathematics anxiety; (b) the association of cognitive activation with mathematics self-efficacy and anxiety was stronger in classrooms with more internal migrant children and (c) mathematics self-efficacy mediated the relationship between cognitive activation and mathematics anxiety.