Skip to main content

Research Repository

See what's under the surface

Advanced Search

Deep imitation learning for 3D navigation tasks.

Hussein, Ahmed; Elyan, Eyad; Gaber, Mohamed Medhat; Jayne, Chrisina

Authors

Ahmed Hussein

Mohamed Medhat Gaber

Chrisina Jayne



Abstract

Deep learning techniques have shown success in learning from raw high dimensional data in various applications. While deep reinforcement learning is recently gaining popularity as a method to train intelligent agents, utilizing deep learning in imitation learning has been scarcely explored. Imitation learning can be an efficient method to teach intelligent agents by providing a set of demonstrations to learn from. However, generalizing to situations that are not represented in the demonstrations can be challenging, especially in 3D environments. In this paper, we propose a deep imitation learning method to learn navigation tasks from demonstrations in a 3D environment. The supervised policy is refined using active learning in order to generalize to unseen situations. This approach is compared to two popular deep reinforcement learning techniques: Deep-Q-networks (DQN) and Asynchronous actor critic (A3C). The proposed method as well as the reinforcement learning methods employ deep convolutional neural networks and learn directly from raw visual input. Methods for combining learning from demonstrations and experience are also investigated. This combination aims to join the generalization ability of learning by experience with the efficiency of learning by imitation. The proposed methods are evaluated on 4 navigation tasks in a 3D simulated environment. Navigation tasks are a typical problem that is relevant to many real applications. They pose the challenge of requiring demonstrations of long trajectories to reach the target and only providing delayed rewards (usually terminal) to the agent. The experiments show that the proposed method can successfully learn navigation tasks from raw visual input while learning from experience methods fail to learn an effiective policy. Moreover, it is shown that active learning can significantly improve the performance of the initially learned policy using a small number of active samples.

Journal Article Type Article
Publication Date Apr 30, 2018
Journal Neural computing and applications
Print ISSN 0941-0643
Electronic ISSN 1433-3058
Publisher Springer (part of Springer Nature)
Peer Reviewed Peer Reviewed
Volume 29
Issue 7
Pages 389-404
Institution Citation HUSSEIN, A., ELYAN, E., GABER, M.M. and JAYNE, C. 2018. Deep imitation learning for 3D navigation tasks. Neural computing and applications [online], 29(7), pages 389-404. Available from: https://doi.org/10.1007/s00521-017-3241-z
DOI https://doi.org/10.1007/s00521-017-3241-z
Keywords Deep learning; Convolutional neural networks; Learning from demonstrations; Reinforcement learning; Active learning; 3D navigation; Benchmarking

Files





You might also like



Downloadable Citations

;