This article proposes a new reinforcement learning (RL)-based adaptive control design for precision motion control of a two-degree-of-freedom piezoelectric XY nanopositioning system. In this design, an actor-critic structure is developed to eliminate the effects of uncertain nonlinearities and cross-coupling motion between the two working axes. Then, an adaptive parameter adjustment mechanism is designed to optimize the control performance without a priori knowledge of the unknown perturbations. The effectiveness and superiority of the proposed method are verified by performing simulation and experimental studies. The results show that the proposed RL-based adaptive control method provides a better robust performance and smaller tracking error for the nanopositioning system.