Massive multiple-input multiple-output (MIMO) is a promising technology for enabling cellu-lar-connected unmanned aerial vehicle (UAV) communications in the future. Equipped with full-dimensional large arrays, ground base stations (GBSs) can apply adaptive fine-grained 3D beamforming to mitigate the strong interference between high-Altitude UAVs and low-Altitude terrestrial users, thus significantly enhancing the network spectral efficiency. However, the performance gain of massive MIMO critically depends on accurate channel state information of both UAVs and terrestrial users at the GBSs, which is practically difficult to achieve due to UAV-in-duced pilot contamination and UAVs' high mobility in 3D. Moreover, the increasingly popular applications relying on a large group of coordinated UAVs or UAV swarm as well as the practical hybrid GBS beamforming architecture for massive MIMO further complicate the pilot contamination and channel/beam tracking problems. In this article, we provide an overview of the above challenging issues, propose new solutions to cope with them, and discuss promising directions for future research. Preliminary simulation results are also provided to validate the effectiveness of the proposed solutions.