Deep imitation learning for 3D navigation tasks.

Hussein, Ahmed; Elyan, Eyad; Gaber, Mohamed Medhat; Jayne, Chrisina

Abstract

Deep learning techniques have shown success in learning from raw high dimensional data in various applications. While deep reinforcement learning is recently gaining popularity as a method to train intelligent agents, utilizing deep learning in imitation learning has been scarcely explored. Imitation learning can be an efficient method to teach intelligent agents by providing a set of demonstrations to learn from. However, generalizing to situations that are not represented in the demonstrations can be challenging, especially in 3D environments. In this paper, we propose a deep imitation learning method to learn navigation tasks from demonstrations in a 3D environment. The supervised policy is refined using active learning in order to generalize to unseen situations. This approach is compared to two popular deep reinforcement learning techniques: Deep-Q-networks (DQN) and Asynchronous actor critic (A3C). The proposed method as well as the reinforcement learning methods employ deep convolutional neural networks and learn directly from raw visual input. Methods for combining learning from demonstrations and experience are also investigated. This combination aims to join the generalization ability of learning by experience with the efficiency of learning by imitation. The proposed methods are evaluated on 4 navigation tasks in a 3D simulated environment. Navigation tasks are a typical problem that is relevant to many real applications. They pose the challenge of requiring demonstrations of long trajectories to reach the target and only providing delayed rewards (usually terminal) to the agent. The experiments show that the proposed method can successfully learn navigation tasks from raw visual input while learning from experience methods fail to learn an effiective policy. Moreover, it is shown that active learning can significantly improve the performance of the initially learned policy using a small number of active samples.

Citation

HUSSEIN, A., ELYAN, E., GABER, M.M. and JAYNE, C. 2018. Deep imitation learning for 3D navigation tasks. Neural computing and applications [online], 29(7), pages 389-404. Available from: https://doi.org/10.1007/s00521-017-3241-z

• Journal Article Type: Article
• Acceptance Date: Oct 4, 2017
• Online Publication Date: Dec 4, 2017
• Publication Date: Apr 30, 2018
• Deposit Date: Oct 10, 2017
• Publicly Available Date: Oct 10, 2017
• Journal: Neural computing and applications
• Print ISSN: 0941-0643
• Electronic ISSN: 1433-3058
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Volume: 29
• Issue: 7
• Pages: 389-404
• DOI: https://doi.org/10.1007/s00521-017-3241-z
• Keywords: Deep learning; Convolutional neural networks; Learning from demonstrations; Reinforcement learning; Active learning; 3D navigation; Benchmarking
• Public URL: http://hdl.handle.net/10059/2543