As a significant component of executive function, goal-directed attentional control is crucial for cognitive processing and is closely linked to frontal-midline theta (FMT) rhythms. However, how up-regulation and down-regulation of FMT through neurofeedback training (NFT) impact goal-directed attention control remains unclear, especially for both short-term and long-lasting effects. Therefore, this study employed a single-blind sham-controlled between-subject design to answer this question. Forty-seven healthy adults were randomly assigned to the up-regulation, down-regulation, or sham control groups. Each group underwent one NFT session per day at the Fz electrode site for four consecutive days. All participants completed a visual search task before, immediately after the first, after the final, and one week following the last NFT session. The down-regulation group significantly reduced FMT activity during NFT and in the resting state (p < = 0.038), while the up-regulation group only showed an upward trend during the training phase (r = 0.721, p = 0.002). The behavioral performance showed no significant improvement in any group (p > 0.05). Importantly, the FMT learning efficacy in the up-regulation group revealed a significantly negative correlation with the change in switch cost (r = -0.602, p = 0.046). These findings suggest a close link between the up-regulation efficacy of FMT rhythms and goal-directed attentional control. In educational or clinical settings, it would be desirable to improve goal-directed attention through enhancement of FMT up-regulation efficacy of NFT in future work.