COMPASS
COMPASS Safety Management in Complex ATM System of Systems using ICT approaches
Contents
About
The project addresses the challenge of how to identify combinations of events as they are happening and issue warnings to operators early enough to take action and prevent catastrophic failures. The project allows for mining of safety patterns from past data, filtering and enriching these patterns using expertise and domain knowledge and then using them to monitor running ATM systems automatically.
Context
ATC is a service provided by ground-based controllers to direct and monitor aircraft through controlled airspace (and on the ground). The primary purpose of ATC systems is to prevent aircraft collisions, and this is achieved by having aircraft maintain a lateral and/or vertical minimum distance (separation minima). Maintaining separation minima requires ATCo to have access to, and interpret, aircraft position data. This data can be obtained through a number of mechanisms, with radar and aircraft position reports being most common.
Oftentimes, ATC are seamlessly able to maintain separation between aircraft. However, because of increasing levels of air traffic and increasing pressure to optimise the use of airspace for economic and environmental reasons, airspace is becoming increasingly difficult to manage and occasionally a loss of separation (conflict) between aircraft may occur. Moreover, upon detecting a conflict, the process of finding a solution may be complex because of the impact that ATC instructions may have on other aircraft not actively involved in the conflict. Early safety warnings, with regard to conflicts, are therefore particularly desirable in the ATC domain. In recognition of this, a number of technologies have been introduced into the ATC domain in an attempt to improve air traffic safety.
- Airborne Collision Avoidance Systems (ACAS) are systems deployed within aircraft to warn pilots of the presence of nearby aircraft, which may present a threat of collision. These systems are therefore designed to augment the activities of ATC, and according to the literature, improve safety in the airspace by a factor of between 3 and 5.
- Short Term Conflict Alert (STCA) is a ground-based safety net operated directly by ATC. It checks possible conflicting aircraft trajectories in a time horizon of about 2 or 3 minutes and alerts the ATCo prior to a conflict.
- Finally, Medium Term Conflict Detection (MTCD) is a supporting technology in ATC, which identifies potential conflicts in a horizon of up to 20 mins. The benefit of this proposed technology is that ATCo would be able to plan solutions to conflicts, which minimises the likelihood of causing further conflicts between either the aircraft involved in the conflict at some future point, or to other independent aircraft. Such scenarios are frequent with STCA since the 2 or 3 minute look-ahead leaves ATCo a very short window of opportunity to analyse the impact of the possible solutions.
Hypothesis and Objectives
The research hypothesis of COMPASS was that safety patterns extracted from historical data can be used to classify and prioritise future safety-related events (e.g. conflicts). The objectives of COMPASS was to extract such safety patterns by mining historical data, to provide an environment that can visualise the traffic crossing ATM en-route sectors and detect instances of these patterns in this traffic, to provide a toolkit for defining artificial operational scenarios, and to provide a distributed environment for assessing the precision of mined safety patterns.
Methodology
To achieve the project's objectives, historical records of the planned and actual trajectory of flights across Europe (ALL_FT) were made available by Eurocontrol. This data was analysed in order to identify trajectories where the flight plan inferred a potential future conflict between aircraft. Furthermore, the data was analysed to identify the actual outcome of any detected potential conflict. In essence, the outcome will either be an actual occurrence of a conflict (either because of inadequate ATC or flight crew action), or no conflict (e.g. because of intervention by the ATC or the flight crew). The former potential conflicts are important to identify for the safe operation of the system, whereas the latter are the “nuisance” false positive potential conflicts.
To identify interesting safety-related events (iEvents) that are more likely to materialise COMPASS employed data mining techniques to construct safety patterns. While mining safety patterns, one of the main ideas that emerged was that the history of flights should somehow be considered in the analysis. In other words, the behaviour of the same flight across different days should provide some useful information, and whenever a repetitive pattern is found, deviations from that pattern are expected to significantly impact the ATC operation. In support of this, COMPASS defined Trajectory Synchronisation Likelihood (TSL) – a measure of the synchronization of the trajectories of two aircraft. Generally speaking, two aircraft are synchronized when both of them present significant deviations from their usual trajectories at the same time. Therefore, and as a first step, it is necessary to define what the usual trajectory of an aircraft is, and how to measure whether a deviation is statistically significant. Suppose a safety related event has been identified on a given day, in which two flights were involved: AIR0001 and AIR0002. The same two flights may have been operated in previous days, and this can be easily checked by examining flights with the same code, operating between the same pair of airports and with the same planned departure time. These other historical flights are then used to compute the expected position of each aircraft at the time of the considered event. The historical flight analysis associated with the TSL presents several major challenges. First of all, one must face the very large quantity of data to be analysed, which requires highly optimized algorithms; this is especially true if a real-time implementation is sought, and therefore results have to be obtained as soon as aircraft make their appearance. Second, it is important to include in the analysis the interactions between different aircraft, and not just consider each trajectory as independent. This, in turn, further aggravates the problem of the computational cost.
Given the high computational cost associated with the initial safety pattern, a reduced safety pattern was mined, which avoids the use of the TSL. As expected, the efficiency of the reduced safety pattern is lower than the one corresponding to the complete pattern, as important factors, like, for example, the ones related with the TSL, have been discarded. In any case, the maximum value of this proportion for the reduced safety pattern is still three times higher than the one observed in the case of random classification, indicating that relevant knowledge is still extracted from the system and that the reduced pattern is still of utility.
To enable domain experts to visualise planned and executed trajectories and classify conflicts using automatically-mined or otherwise constructed safety patterns, the Early Safety Warning System (ESWS) was developed to provide a GUI workbench that allows geographical data to be plotted onto an interactive map (selectable, allowing navigation and resizing of the map). This allows the visualisation of the ESWS model, which provides a representation of objects, which play a prominent part in air traffic management. Examples of objects of the ESWS model include airspaces, airways, and flight trajectories. To build this model, the ESWS reads data from a database pre-populated with a sample of the ALL_FT dataset. The significance of the ESWS is that it enables Air Traffic Control (ATC) to perform various types of analyses of airspace such as occupation analysis, important point analysis, air fragment analysis and conflict analysis and classification. Multiple panels in the ESWS GUI workbench allow easy access and viewing of the results of these analyses. This enables the intuitive presentation of relevant information and analysis results to ATM experts. Conflict analysis and classification capabilities give ATC an early indication of potentially conflicting aircraft trajectories, which may result in the aircraft crossing a point with inadequate separation. To enable domain experts to experiment with artificial operational scenarios, COMPASS defined an Operational Scenario Language (OSL) and supporting tools (editors, support from initialising scenarios from real data) integrated with the ESWS.
As discussed above, ESWS was developed as a desktop-based application with a rich user interface. While a user interface is necessary in scenarios that involve user interaction, it is unnecessary – and arguably detrimental from a performance and automation perspective – when batch analysis needs to be performed to evaluate the accuracy (precision and recall) of safety patterns across a large number of days and airspaces (sectors). To enable the ESWS to evaluate the reduced safety pattern against such large datasets, it was necessary to re-engineer the system into a scalable form that could operate on a distributed computing environment without user interaction. After re-engineering the system as a distributed service-oriented architecture, the reduced safety pattern was evaluated against a subset of the ALL_FT dataset. The approach that was taken in selecting data samples was to randomly select dates from each month of the aforementioned data range thus providing a certain level of even coverage. This strategy was chosen since seasonal variations in air traffic occur and it is important that the ESWS is evaluated in the context of such variations. Eventually, 44 dates were selected, representing nearly 15% of the complete dataset.
Results
The prediction outcome results show that the reduced safety pattern has an average correct prediction accuracy rate of 86%, a false positive rate of 5% and a 9% false negative rate. Moreover, these values only deviate by 4% across all of the dates that the pattern was evaluated with. The results from the experiments suggest that the reduced safety pattern is consistently accurate. The high number of correct predictions outcomes, low number of false positives outcomes, and low average deviation of predicted conflict occurrence time across different airspaces, different dates of the year and even different times of day demonstrates the relevance of the mined safety pattern. The unanticipated number of false negative prediction outcome results and the fairly high maximum deviation of predicted conflict occurrence time are not believed to pose a threat to the safety of airspace. Systems such as ESWS are in use to help ATM experts to plan their actions well in advance of the predicted iEvents. Data from a multitude of systems and procedures is used to shape the final decisions and actions of ATCo. Clearly it is inevitable that looking ahead and predicting aircraft trajectories and conflicts by 60 minutes or so will have a certain level of uncertainty attached to it. Given that the ESWS will be allowed to refine its results as conflicts approach, and given that the ESWS is designed not to replace STCA or other systems but instead to augment them, the false negative rate is not deemed to be an obstacle in the adoption of the ESWS to the ATC domain.
Future Research Lines and conclusions
Future research directions include investigating the value of sector-specific patterns, integrating additional sources of information in the pattern mining process, and applying the data-driven approach followed in COMPASS for the detection of patterns in other types of sectors (e.g. Terminal Sectors). Exploring the particular phases of the ATM process (e.g. capacity management, traffic complexity management) in which an early safety warning system such as the one proposed by COMPASS could realistically fit in is also an area for further exploration.
Publications
- M. Zanin, D. Perez, K. Chatterjee, D. Kolovos, R. Paige, A. Horst and B. Rumpe. On Demand Data Analysis and Filtering for Inaccurate Flight Trajectories. In: Schaefer, Dirk (ed) Proceedings of the SESAR Innovation Days (2011) EUROCONTROL. ISBN 978-2-87497-024-5.
- M. Zanin, Synchronization Likelihood in Aircraft Trajectories, in Proceedings of the Tenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2013), Chicago, 2013.
Technical Video
Research Team
THALES Information Systems (Project co-ordinator), University of York (Technical co-ordinator), INNAXIS Research Institute, RWTH Aachen University
