Trajectory tracking control of autonomous trolley collection robots (ATCR) is a challenging task due to the complex environment, significant noise, and external disturbances. To address these challenges, this work investigates a control scheme for ATCRs subject to severe environmental interference. A fast-convergent, kinematics model-based adaptive sliding mode disturbance observer (ASMDOB) is first proposed to estimate lumped disturbances. Building upon this, a robust controller with prescribed performance is designed using the backstepping technique, improving transient performance and guaranteeing fast convergence. Simulation results have been given to illustrate the effectiveness of the proposed control scheme.