Mechanics-Based Algorithms for Sampling, Control, and Learning in Non-Convex Domains

This grant will fund research that enables smart devices to adapt automatically to novel situations, with reliable guarantees of safe and efficient behavior, thereby promoting the progress of science and advancing the national prosperity and health. In the near future, a large portion of the population will rely on devices such as self-driving cars and smart medical implants that make safety-critical decisions without human intervention. In these devices, it is not possible for engineers to manually prescribe all the behaviors that will arise during operation. A self-driving car must steer reliably in unfamiliar road conditions. A neural stimulator for seizure suppression must be personalized to the individual patient. To enable deployment of highly autonomous smart devices on a large scale, these devices must be able to learn appropriate behaviors by themselves. Currently, most learning algorithms for real-world automated systems lack provable guarantees for safety and performance. The methods devised in this project will overcome this limitation, benefitting applications in transportation, healthcare, and home automation. The development of remotely controllable physical experiments will help make principles of control and automated learning accessible to high school and undergraduate students.

This research aims to make fundamental contributions to the development of a model-based reinforcement learning methodology that guarantees stability and near-optimal performance for a wide class of unknown nonlinear stochastic systems. It achieves this outcome by addressing two unresolved challenges for existing learning methods: 1) a lack of provable guarantees for convergence to desired probability distributions and to global optima of the corresponding non-convex optimization problems, and 2) the lack of stability guarantees or need for initial stabilizing controllers. The research will leverage new insights on non-smooth stochastic processes to quantitatively bound convergence of solutions around global optima for a collection of algorithms derived from mechanics. Stabilizing controllers for nonlinear stochastic systems will be obtained by a novel variation on the policy iteration method, without requiring an initial stabilizing controller. The work will contribute to a rigorous understanding of algorithms for sampling, optimization, and learning for non-convex losses in non-convex domains, as well as methods of control policy evaluation, stability verification, and optimization.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.