FlyBeam: Echo State Learning for Joint Flight and Beamforming Control in Wireless UAV Networks

This paper aims at designing high-data-rate swarm UAV networks with distributed beamforming capabilities. The primary challenge is that the beamforming gain in swarm UAV networks is highly affected by the UAVs' flight altitude, their movements and the resulting intermittent link blockages, as well as the availability of channel state information (CSI) at individual UAVs. To address this challenge, we propose FlyBeam, a learning- based framework for joint flight and beamforming control in swarm UAV networks. We first present a mathematical formulation of the control problem with the objective of maximizing the throughput of swarm UAV networks by jointly controlling the flight and distributed beamforming of UAVs. Then, a distributed solution algorithm is designed based on a combination of Echo State Network learning and online reinforcement learning. The former is adopted to approximate the utility function for individual UAVs based on online measurements, by jointly considering the unknown blockage dynamics and other factors that affect the beamforming gain. The latter is used to guide the exploitation and exploration in FlyBeam. The effectiveness of FlyBeam is evaluated through an extensive simulation campaign. Results indicate that significant (up to 450%) beamforming gain can be achieved by FlyBeam. We also investigate the effects of blockages and UAV flight altitude on the beamforming gain. It is found that, which is somewhat surprising, higher (rather than lower) beamforming gain can be achieved by FlyBeam with denser blockages in swarm UAV networks.