Federated Learning (FL) enables the distributed machine learning (ML) without violating the privacy of local users. In the scenario wireless FL, it is challenging for some local clients to establish reliable connections with the parameter server due to the potential long-distance transmission. To address this issue, unmanned aerial vehicle (UAV) can be leveraged as a relay between the FL parameter server and local clients for efficiently forwarding the ML models. In this work, we propose a two-tier hierarchical FL scheme assisted by a UAV swarm. During the local training phase, the UAVs offload their own data to the base station (BS). For the remaining time, the UAVs act as the relays to assist the parameter server and local clients in forwarding ML models. To optimize the FL convergence and the UAVs' data transmissions, we formulate a joint optimization of the matching between the UAVs and local clients, the time allocation of the hierarchical FL, and the number of iterations for the local model training. To solve this optimization problem, we design an efficient algorithm that integrates a subgradient-based method with the cross entropy-based genetic algorithm. Numerical results are provided to demonstrate the advantages of our proposed two-tier hierarchical FL scheme with the UAV swarm and our proposed algorithm.