“Safety implications of autonomous vehicles” wins Best Paper at 2023 Symposium of the Society of Flight Test Engineers

John Thomas, co-director of the Engineering Systems Lab, along with flight test engineer Ryan Bowers, were awarded Best Paper at the 54th Annual International Symposium of the Society of Flight Test Engineers, October 2023.

Abstract
This paper investigates the safety implications of flight testing an Uncrewed Air Vehicle (UAV) controlled by a neural network-based flight autonomy software, and the utility of System Theoretic Process Analysis (STPA) in identifying risks. The host UAV in this case study includes various control regimes and handoffs over the course of a sortie including human control, traditional autopilot, and an artificial intelligence autonomy software trained using Deep Reinforcement Learning (DRL) machine learning techniques. The flight test operational environment includes flight in both civil and restricted airspace, and at least one nearby crewed chase aircraft to observe the UAV in flight. STPA was applied after traditional airworthiness and safety assessment processes but before flight test to identify and mitigate potential new hazards associated with the UAV technology and its operation. STPA was found to identify new risks, vulnerabilities, and undocumented assumptions that were used to create practical improvements in the technology, operational planning, and flight test. STPA produced additional mitigations related to the UAV, the automated run-time assurance mechanisms, human controls and other interactions with the UAV, and the overall operation of the autonomy. This paper summarizes some of the additional critical findings discovered by STPA prior to flight test, including:
1. The autonomy command limiters would not prevent unsafe combinations of control inputs that are individually within limits. Once STPA identified this gap, new mitigations were created to address it.
2. The original UAV safety mechanisms could not be easily modified to enforce the cleared envelope, so a new envelope protection mechanism was needed.
3. The original design of the human/autonomy handoff introduced potentially catastrophic scenarios related to confusion over whether the human pilot or the autonomy was in control of the host UAV. New mitigations were proposed to address these scenarios.
4. The UAV possessed a scripted maneuver designed to safely transition the UAV from autonomy control to human control, but that maneuver introduced new unforeseen risks.