Abstract
The two leading solution paradigms for imitation learning (IL), BC and GAIL, each suffers from notable drawbacks. BC, a supervised learning approach to mimic expert actions, is vulnerable to covariate shift. GAIL applies adversarial training to minimize the discrepancy between expert and learner behaviors, which is prone to unstable training and mode collapse. In this work, we propose DC - Distributional Cloning - a novel IL approach for addressing the covariate shift and mode collapse problems simultaneously. DC directly maximizes the likelihood of observed expert and learner demonstrations, and gradually encourages the learner to evolve towards expert behaviors based on an averaging effect. The DC solution framework contains two stages in each training loop, where in stage one the mixed expert and learner state distribution is estimated via SoftFlow, and in stage two the learner policy is trained to match both the expert's policy and state distribution via ADMM. Experimental evaluation of DC compared with several baselines in 10 different physics-based control tasks reveal superior results in learner policy performance, training stability, and mode distribution preservation.
Original language | English (US) |
---|---|
Title of host publication | Proceedings - 23rd IEEE International Conference on Data Mining, ICDM 2023 |
Editors | Guihai Chen, Latifur Khan, Xiaofeng Gao, Meikang Qiu, Witold Pedrycz, Xindong Wu |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 818-827 |
Number of pages | 10 |
ISBN (Electronic) | 9798350307887 |
DOIs | |
State | Published - 2023 |
Event | 23rd IEEE International Conference on Data Mining, ICDM 2023 - Shanghai, China Duration: Dec 1 2023 → Dec 4 2023 |
Publication series
Name | Proceedings - IEEE International Conference on Data Mining, ICDM |
---|---|
ISSN (Print) | 1550-4786 |
Conference
Conference | 23rd IEEE International Conference on Data Mining, ICDM 2023 |
---|---|
Country/Territory | China |
City | Shanghai |
Period | 12/1/23 → 12/4/23 |
Bibliographical note
Publisher Copyright:© 2023 IEEE.
Keywords
- imitation learning
- neural ordinary differential equations