An unsupervised autonomous learning framework for goal-directed behaviours in dynamic contexts.

Ezenkwu, Chinedu Pascal; Starkey, Andrew

Abstract

Due to their dependence on a task-specific reward function, reinforcement learning agents are ineffective at responding to a dynamic goal or environment. This paper seeks to overcome this limitation of traditional reinforcement learning through a task-agnostic, self-organising autonomous agent framework. The proposed algorithm is a hybrid of TMGWR for self-adaptive learning of sensorimotor maps and value iteration for goal-directed planning. TMGWR has been previously demonstrated to overcome the problems associated with competing sensorimotor techniques such SOM, GNG, and GWR; these problems include: difficulty in setting a suitable number of neurons for a task, inflexibility, the inability to cope with non-markovian environments, challenges with noise, and inappropriate representation of sensory observations and actions together. However, the binary sensorimotor-link implementation in the original TMGWR enables catastrophic forgetting when the agent experiences changes in the task and it is therefore not suitable for self-adaptive learning. A new sensorimotor-link update rule is presented in this paper to enable the adaptation of the sensorimotor map to new experiences. This paper has demonstrated that the TMGWR-based algorithm has better sample efficiency than model-free reinforcement learning and better self-adaptivity than both the model-free and the traditional model-based reinforcement learning algorithms. Moreover, the algorithm has been demonstrated to give the lowest overall computational cost when compared to traditional reinforcement learning algorithms.

Citation

EZENKWU, C.P. and STARKEY, A. 2022. An unsupervised autonomous learning framework for goal-directed behaviours in dynamic contexts. Advances in computational intelligence [online], 2(3), article number 26. Available from: https://doi.org/10.1007/s43674-022-00037-9

• Journal Article Type: Review
• Acceptance Date: Apr 27, 2022
• Online Publication Date: Jun 2, 2022
• Publication Date: Jun 30, 2022
• Deposit Date: Mar 29, 2024
• Publicly Available Date: Apr 25, 2024
• Journal: Advances in computational intelligence
• Print ISSN: 2730-7794
• Electronic ISSN: 2730-7808
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Volume: 2
• Issue: 3
• Article Number: 26
• DOI: https://doi.org/10.1007/s43674-022-00037-9
• Keywords: Autonomous agents; Artificial intelligence; Machine learning
• Public URL: https://rgu-repository.worktribe.com/output/2288050