Deep imitation learning for 3D navigation tasks.
Hussein, Ahmed; Elyan, Eyad; Gaber, Mohamed Medhat; Jayne, Chrisina
Doctor Eyad Elyan email@example.com
Mohamed Medhat Gaber
Deep learning techniques have shown success in learning from raw high dimensional data in various applications. While deep reinforcement learning is recently gaining popularity as a method to train intelligent agents, utilizing deep learning in imitation learning has been scarcely explored. Imitation learning can be an efficient method to teach intelligent agents by providing a set of demonstrations to learn from. However, generalizing to situations that are not represented in the demonstrations can be challenging, especially in 3D environments. In this paper, we propose a deep imitation learning method to learn navigation tasks from demonstrations in a 3D environment. The supervised policy is refined using active learning in order to generalize to unseen situations. This approach is compared to two popular deep reinforcement learning techniques: Deep-Q-networks (DQN) and Asynchronous actor critic (A3C). The proposed method as well as the reinforcement learning methods employ deep convolutional neural networks and learn directly from raw visual input. Methods for combining learning from demonstrations and experience are also investigated. This combination aims to join the generalization ability of learning by experience with the efficiency of learning by imitation. The proposed methods are evaluated on 4 navigation tasks in a 3D simulated environment. Navigation tasks are a typical problem that is relevant to many real applications. They pose the challenge of requiring demonstrations of long trajectories to reach the target and only providing delayed rewards (usually terminal) to the agent. The experiments show that the proposed method can successfully learn navigation tasks from raw visual input while learning from experience methods fail to learn an effiective policy. Moreover, it is shown that active learning can significantly improve the performance of the initially learned policy using a small number of active samples.
|Journal Article Type||Article|
|Publication Date||Apr 30, 2018|
|Journal||Neural computing and applications|
|Publisher||Springer (part of Springer Nature)|
|Peer Reviewed||Peer Reviewed|
|Institution Citation||HUSSEIN, A., ELYAN, E., GABER, M.M. and JAYNE, C. 2018. Deep imitation learning for 3D navigation tasks. Neural computing and applications [online], 29(7), pages 389-404. Available from: https://doi.org/10.1007/s00521-017-3241-z|
|Keywords||Deep learning; Convolutional neural networks; Learning from demonstrations; Reinforcement learning; Active learning; 3D navigation; Benchmarking|
HUSSEIN 2018 Deep imitation learning for 3D
You might also like
Data stream mining: methods and challenges for handling concept drift.
Multiple fake classes GAN for data augmentation in face image dataset.
Digitisation of assets from the oil and gas industry: challenges and opportunities.
Neighbourhood-based undersampling approach for handling imbalanced and overlapped data.