Prof. Jon How @ Rosenbrock Lecture Series

Keynote by Prof. Jon How | Efficient, Agile, Data-driven Vision-Based Onboard Autonomy

Real-world, large-scale, vision-based multi-agent autonomy demands the ability to efficiently sense, plan, and act under uncertainties. Although vision-based data is a rich and relatively easily acquired source of information, perceptual uncertainties and constraints—such as limited fields-of-view in planning, as well as onboard computational and communication limits —necessitate careful consideration at the algorithmic level. In this talk, we present strategies to account for vision-based constraints at the control, planning, and localization levels. First, we introduce a strategy to enable computationally efficient learning of vision-based neural networks for control using Imitation Learning. Our method leverages the properties of a robust tube model predictive controller to collect data in a way that is data- and computation-efficient, requiring only a single demonstration, while accounting for the effects of real world uncertainties.