Active eavesdropper sends jamming signals to raise the transmit power of base stations and steal more information from cellular systems. Network coding resists the active eavesdroppers that cannot obtain all the data flows, but highly relies on the wiretap channel states that are rarely known in wireless networks. In this paper, we present a reinforcement learning (RL) based random linear network coding scheme for drone-aided cellular systems to address eavesdropping. In this scheme, the network coding policy, including the encoded packet number, the packet and power allocation, is chosen based on the measured jamming power, previous transmission performance and BS channel states. A virtual model generates simulated experiences to update Q-values besides real experiences for faster policy optimization. We also propose a deep RL version and design a hierarchical architecture to further accelerate the policy exploration and improve the anti-eavesdropping performance, in terms of the intercept probability, the latency, the outage probability and the energy consumption. We analyze the computational complexity, drone deployment, secure coverage area and the performance bound of the proposed schemes, which are verified via simulation results.