Skip to main content

Research Repository

Advanced Search

Deep imitation learning with memory for robocup soccer simulation.

Hussein, Ahmed; Elyan, Eyad; Jayne, Chrisina

Authors

Ahmed Hussein

Chrisina Jayne



Contributors

Elias Pimenidis
Editor

Chrisina Jayne
Editor

Abstract

Imitation learning is a field that is rapidly gaining attention due to its relevance to many autonomous agent applications. Providing demonstrations of effective behaviour to teach the agent is useful in real world challenges such as sparse rewards and dynamic environments. However, most imitation learning approaches don't retain a memory of previous actions and treat the demonstrations as independent and identically distributed samples. This neglects the temporal dependency between low-level actions that are performed in sequence to achieve the desired behaviour. This paper proposes an imitation learning method to learn sequences of actions by utilizing memory in deep neural networks. Long short-term memory networks are utilized to capture the temporal dependencies in a teacher's demonstrations. This way, past states and actions provide context for performing following actions. The network is trained using raw low-level features and directly maps the input to low-level parametrized actions in real-time. This minimizes the need for task specific knowledge to be manually employed in the learning process compared to related approaches. The proposed methods are evaluated on a benchmark soccer simulator and compared to supervised learning and data-aggregation approaches. The results show that utilizing memory while learning significantly improves the performance and generalization of the agent and can provide a stationary policy than can produce robust predictions at any point in the sequence.

Citation

HUSSEIN, A., ELYAN, E. and JAYNE, C. 2018. Deep imitation learning with memory for robocup soccer simulation. In Pimenidis, E. and Jayne, C. (eds.) Proceedings of the 19th International conference on engineering applications of neural networks (EANN 2018), 3-5 September 2018, Bristol, UK. Communications in computer and information science, 893. Cham: Springer [online], pages 31-43. Available from: https://doi.org/10.1007/978-3-319-98204-5_3

Conference Name 19th International conference on engineering applications of neural networks (EANN 2018)
Conference Location Bristol, UK
Start Date Sep 3, 2018
End Date Sep 5, 2018
Acceptance Date May 31, 2018
Online Publication Date Jul 27, 2018
Publication Date Jul 27, 2018
Deposit Date Oct 16, 2018
Publicly Available Date Jul 28, 2019
Print ISSN 1865-0929
Electronic ISSN 1865-0937
Publisher Springer
Pages 31-43
Series Title Communications in computer and information science
Series Number 893
Series ISSN 1865-0937
ISBN 9783319982038
DOI https://doi.org/10.1007/978-3-319-98204-5_3
Keywords Imitation learning; Sequences; Deep neural networks
Public URL http://hdl.handle.net/10059/3174

Files





You might also like



Downloadable Citations