Learning to stabilize high-dimensional unknown systems using lyapunov-guided exploration

Songyuan Zhang and Prof. Chuchu Fan (REALM) developed an approach for learning stabilizing controllers for high-dimensional unknown systems. They used experiments, including a high-fidelity F-16 fighter jet simulation model with a ground collision avoidance system, to demonstrate the scalability of their algorithm and prove that their algorithm can be extended to learn controllers guided by alternative certificate functions for unknown systems.

Authors: Songyuan Zhang, Chuchu Fan
Citation: Proceedings of Machine Learning Research vol vvv:1–32, 2024

Abstract:
Designing stabilizing controllers is a fundamental challenge in autonomous systems, particularly for high-dimensional, nonlinear systems that cannot be accurately modeled using differential equations. Lyapunov theory offers a robust solution for stabilizing control systems. Still, current methods relying on Lyapunov functions require access to complete dynamics or samples of system executions throughout the entire state space. Consequently, they are impractical for high-dimensional systems. In this paper, we introduce a novel framework, LYGE, for learning stabilizing controllers specifically tailored to high-dimensional, unknown systems. LYGE employs Lyapunov theory to iteratively guide the search for samples during exploration while simultaneously learning the local system dynamics, control policy, and Lyapunov functions. We demonstrate its scalability on highly complex systems, including a high-fidelity F-16 jet model from the Air Force featuring a 16D state space and a 4D input space. Experimental results indicate that, compared to prior works in reinforcement learning, imitation learning, and neural certificates, LYGE reduces the distance to the goal by 50% while requiring only 5% to 32% of the samples. Furthermore, we demonstrate that our algorithm can be extended to learn controllers guided by alternative certificate functions for unknown systems.