Supporting Simulation Pilots by Automatic Speech Recognition and Understanding

Systems like Alexa, Siri or the Google Assistant that recognize human speech have changed our daily lives during the last decade. Prototypic applications based around speech have since then also found their way into the air traffic management (ATM) domain. Recently pre-filling radar label entries by automatically understanding the air traffic controller to pilot communication has reached the technology readiness level before industrialization (TRL6). DLR is one of the main drivers of such speech-based technologies in the context of ATM. This report addresses an automatic speech recognition and understanding (ASRU) application to support simulation pilots during Human-in-the-Loop experiments. For this purpose, an ASRU system recognizes the verbal clearances of an air traffic controller and forwards the information to the visual interfaces of the human simulation pilots. The pilots confirm the information or make modifications in case of misrecognitions and send it to the simulator for execution. With this approach more than 75% of the commands from the air traffic controller, which the simulation pilot normally has to enter manually, are already recognized by ASRU and the simulation pilot just needs to confirm or modify the ASRU outputs. This dramatically reduces the simulation pilot workload. The remaining 25% of the commands are, however, a challenge. These often contain seldom spoken words related to the airspace, which are relevant in the ATM context. If those commands can also be recognized, more complex simulations are possible with less simulation pilots. This report therefore also presents first results on adjusting ASRU to these seldom spoken words, which are often waypoint names as a part of direct-to clearances, e.g, “mobsa”, “ekern”.