Although deep learning methods have made significant breakthroughs in generic object detection, their performance on aerial images is not satisfactory. Unlike generic images, aerial images have smaller object relative scales (ORS), more low-resolution objects, and serious object scale diversity. Most researches focus on modifying network structures to address these challenges, while few studies pay attention to data enhancement which can be used in combination with model modification to further improve detection accuracy. In this work, a novel data enhancement method called scale adaptive image cropping (SAIC) is proposed to address these three challenges. Specifically, SAIC consists three steps: ORS estimation in which a specific neural network is designed to estimate ORS levels of images; image resizing in which a GAN-based super-resolution method is adopted to up-sample images with the smallest ORS level, easing low-resolution object detection; image cropping in which three cropping strategies are proposed to crop resized images, adjusting ORS. Extensive experiments are conducted to demonstrate the effectiveness of our method. SAIC improves the accuracy of feature pyramid network (FPN) by 9.65% (or relatively 37.06%). Without any major modification, FPN trained with SAIC won the 3rd rank on 2018 VisDrone challenge detection task.