Skip to main content

Aircraft Trajectory Planning for Climate Hotspot Avoidance Considering Air Traffic Complexity: A Constrained Multi-Agent Reinforcement Learning Approach

Paper ID

SIDs-2024-084

Conference

SESAR Innovation Days

Year

2024

Theme

Trajectory and arrival management

Project Name

Horizon Europe project REFMAP

Keywords:

climate impact; aircraft trajectory optimization; air traffic management system; multi-agent reinforcement learning; constrained Markov decision process; proximal policy optimization algorithm

Authors

Fateme Baneshi, María Cerezo-Magaña, Manuel Soler, Tingting Ni and Maryam Kamgarpour

DOI

https://doi.org/10.61009/SID.2024.1.41

Project Number

101096698

Abstract

Planning aircraft trajectories to avoid climate-sensitive areas poses operational challenges, including increased traffic complexity and potential safety risks. This study presents a framework designed to plan operationally feasible climate-friendly routes from the perspective of the air traffic management (ATM) system. The problem is formulated as a constrained Markov game, where air traffic complexity, a key indicator of air traffic manageability, serves as the objective function, and climate hotspot avoidance is imposed as a constraint. The proposed method employs the multi-agent proximal policy optimization algorithm and adapts it to handle constraints related to climate hotspot avoidance using the Lagrangian technique. To ensure scalability, parameter sharing is employed, allowing the algorithm to deal with varying numbers of concurrently operating aircraft in different scenarios. Experimental results demonstrate that the proposed algorithm effectively balances environmental goals with traffic manageability, offering operationally feasible climate-optimal trajectories.