Deep reward shaping from demonstrations.
Hussein, Ahmed; Elyan, Eyad; Gaber, Mohamed Medhat; Jayne, Chrisina
Doctor Eyad Elyan firstname.lastname@example.org
Mohamed Medhat Gaber
Deep reinforcement learning is rapidly gaining attention due to recent successes in a variety of problems. The combination of deep learning and reinforcement learning allows for a generic learning process that does not consider specific knowledge of the task. However, learning from scratch becomes more difficult when tasks involve long trajectories with delayed rewards. The chances of finding the rewards using trial and error become much smaller compared to tasks where the agent continuously interacts with the environment. This is the case in many real life applications which poses a limitation to current methods. In this paper we propose a novel method for combining learning from demonstrations and experience to expedite and improve deep reinforcement learning. Demonstrations from a teacher are used to shape a potential reward function by training a deep supervised convolutional neural network. The shaped function is added to the reward function used in deep-Q-learning (DQN) to perform off-policy training through trial and error. The proposed method is demonstrated on navigation tasks that are learned from raw pixels without utilizing any knowledge of the problem. Navigation tasks represent a typical AI problem that is relevant to many real applications and where only delayed rewards (usually terminal) are available to the agent. The results show that using the proposed shaped rewards significantly improves the performance of the agent over standard DQN. This improvement is more pronounced the sparser the rewards are.
|Start Date||May 14, 2017|
|Publication Date||Jul 3, 2017|
|Publisher||Institute of Electrical and Electronics Engineers|
|Institution Citation||HUSSEIN, A., ELYAN, E., GABER, M.M. and JAYNE, C. 2017. Deep reward shaping from demonstrations. In Proceedings of the 2017 International joint conference on neural networks (IJCNN 2017), 14-19 May 2017, Anchorage, USA. Piscataway: IEEE [online], article number 7965896, pages 510-517. Available from: https://doi.org/10.1109/IJCNN.2017.7965896|
|Keywords||Deep reinforcement; Reinforcement learning; Generic learning process; DeepQlearning (DQN)|
HUSSEIN 2016 Deep reward shaping
You might also like
Data stream mining: methods and challenges for handling concept drift.
Multiple fake classes GAN for data augmentation in face image dataset.
Digitisation of assets from the oil and gas industry: challenges and opportunities.
Neighbourhood-based undersampling approach for handling imbalanced and overlapped data.