Abstract
We study the problem of generating control laws for systems with unknown dynamics. Our approach is to represent the controller and the value function with neural networks, and to train them using loss functions adapted from the Hamilton-Jacobi-Bellman (HJB) equations. In the absence of a known dynamics model, our method first learns the state transitions from data collected by interacting with the system in an offline process. The learned transition function is then integrated to the HJB equations and used to forward simulate the control signals produced by our controller in a feedback loop. In contrast to trajectory optimization methods that optimize the controller for a single initial state, our controller can generate near-optimal control signals for initial states from a large portion of the state space. Compared to recent model-based reinforcement learning algorithms, we show that our method is more sample efficient and trains faster by an order of magnitude. We demonstrate our method in a number of tasks, including the control of a quadrotor with 12 state variables.
Original language | English (US) |
---|---|
Title of host publication | Proceedings - ICRA 2023 |
Subtitle of host publication | IEEE International Conference on Robotics and Automation |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 953-960 |
Number of pages | 8 |
ISBN (Electronic) | 9798350323658 |
DOIs | |
State | Published - 2023 |
Externally published | Yes |
Event | 2023 IEEE International Conference on Robotics and Automation, ICRA 2023 - London, United Kingdom Duration: May 29 2023 → Jun 2 2023 |
Publication series
Name | Proceedings - IEEE International Conference on Robotics and Automation |
---|---|
Volume | 2023-May |
ISSN (Print) | 1050-4729 |
Conference
Conference | 2023 IEEE International Conference on Robotics and Automation, ICRA 2023 |
---|---|
Country/Territory | United Kingdom |
City | London |
Period | 5/29/23 → 6/2/23 |
Bibliographical note
Publisher Copyright:© 2023 IEEE.