P5. Towards Unified Air Traffic Complexity Management through Graph Theory and Artificial Intelligence

2 – Data-driven trajectory prediction

Air Traffic Management’s (ATM) aim is to ensure separation management of aircraft in an efficient way, minimizing possible delays and costs. The expected increase in air traffic demand across manned and unmanned traffic requires a higher level of automation to support the decision making. Adaptive self-Governed aerial Ecosystem by Negotiated Traffic (AGENT) was an exploratory research project supported by the H2020 Research and Innovation Programme, which proposed a system where the avoidance of potential loss of separations is done in a distributed and collaborative way while the controllers monitor the process. This PhD project is built on AGENT’s future work proposals and seeks possible improvement of several critical aspects of the system through the application of Machine Learning (ML) techniques. There were two clear goals in this project: define airspace complexity in a way that challenges current definitions and overcomes their limitations and investigate how ML can be applied to safety in aviation. We investigate these problems in en-route traffic at the tactical level, as well as UAV systems.

The first major contribution of this thesis has been modelling air traffic as a graph in the context of airspace complexity and conflict resolution. We define a graph with aircraft as nodes and interdependencies between them as edges of the graph. This definition allows for problem specific definitions of interdependencies. We further extend the definition of air traffic as a graph by including the time domain, which creates dynamic graphs. We define airspace complexity as graph connectivity and propose four indicators that combine different topological information and the severity of interdependencies to give a complete and nuanced picture of complexity. These indicators are able to provide a dynamic evolution of complexity by leveraging the modelling choice of air traffic as a dynamic graph. Simulation results indicated that the indicators we propose give detailed information and overcome drawbacks of existing metrics. We evaluated our approach using real and synthetic traffic and demonstrated that the indicators express different facets of complexity, confirming that all indicators are needed. The way we define complexity also provides a new framework in the design of conflict resolution algorithms which considers the reduction of airspace complexity in addition to safety preservations. Conflict Resolution (CR) algorithms could be discouraged from providing solutions that increase the overall complexity of the airspace.

Furthermore, we model CR as Multiagent Reinforcement Learning Problem (MARL). We initially investigate CR only in a pairwise setting using Multiagent Deep Deterministic Policy Gradient (MADDPG) as a learning algorithm. We propose a novel state representation that combines positional information with speed and heading of the aircraft. Additionally, we propose a reward function that not only guides agents towards solving the conflict but also to consider factors such as fuel consumption, airspace complexity and delays. Our results indicate that the agents are capable of solving the conflicts and further learning desired behaviours such as solving them as soon as possible with minimal manoeuvres. However, this method suffers from issues of scalability and nonstationarity. In order to overcome these issues, we utilize Graph Neural Networks (GNNs). GNNs inherently allow communication between agents which facilitates cooperation between them. We apply Graph Convolutional Reinforcement Learning (DGN) in CR for Unmanned Aerial Vehicles (UAV) to solve conflicts with 3 and 4 present aircraft which we assume to be cooperative. We achieve impressive performance with the agents being able to always solve the conflicts. Furthermore, they learn a strategy that increases the distance between them, without previous knowledge of the environment. Currently, we are using this application domain to investigate some fundamental questions in MARL such as agent coordination, heterogeneity and transparency in environments where agents have individual and common goals.