Deep imitation learning with memory for robocup soccer simulation.
Hussein, Ahmed; Elyan, Eyad; Jayne, Chrisina
Professor Eyad Elyan email@example.com
Imitation learning is a field that is rapidly gaining attention due to its relevance to many autonomous agent applications. Providing demonstrations of effective behaviour to teach the agent is useful in real world challenges such as sparse rewards and dynamic environments. However, most imitation learning approaches don't retain a memory of previous actions and treat the demonstrations as independent and identically distributed samples. This neglects the temporal dependency between low-level actions that are performed in sequence to achieve the desired behaviour. This paper proposes an imitation learning method to learn sequences of actions by utilizing memory in deep neural networks. Long short-term memory networks are utilized to capture the temporal dependencies in a teacher's demonstrations. This way, past states and actions provide context for performing following actions. The network is trained using raw low-level features and directly maps the input to low-level parametrized actions in real-time. This minimizes the need for task specific knowledge to be manually employed in the learning process compared to related approaches. The proposed methods are evaluated on a benchmark soccer simulator and compared to supervised learning and data-aggregation approaches. The results show that utilizing memory while learning significantly improves the performance and generalization of the agent and can provide a stationary policy than can produce robust predictions at any point in the sequence.
HUSSEIN, A., ELYAN, E. and JAYNE, C. 2018. Deep imitation learning with memory for robocup soccer simulation. In Pimenidis, E. and Jayne, C. (eds.) Proceedings of the 19th International conference on engineering applications of neural networks (EANN 2018), 3-5 September 2018, Bristol, UK. Communications in computer and information science, 893. Cham: Springer [online], pages 31-43. Available from: https://doi.org/10.1007/978-3-319-98204-5_3
|Conference Name||19th International conference on engineering applications of neural networks (EANN 2018)|
|Conference Location||Bristol, UK|
|Start Date||Sep 3, 2018|
|End Date||Sep 5, 2018|
|Acceptance Date||May 31, 2018|
|Online Publication Date||Jul 27, 2018|
|Publication Date||Jul 27, 2018|
|Deposit Date||Oct 16, 2018|
|Publicly Available Date||Oct 16, 2018|
|Series Title||Communications in computer and information science|
|Keywords||Imitation learning; Sequences; Deep neural networks|
HUSSEIN 2018 Deep imitation learning
Publisher Licence URL
You might also like
Deep learning based approaches for imitation learning.
Deep imitation learning for 3D navigation tasks.
Deep reward shaping from demonstrations.
Imitation learning: a survey of learning methods.
Deep active learning for autonomous navigation.