Using Representation Learning for Scalable Multi-Agent Reinforcement Learning in Heterogeneous Multi-Agent Systems
Summary
In multi-agent tasks with heterogeneous agents, effective solutions may rely on the ability of agents to behave differently. While such heterogeneous multi-agent systems are common, only a minority of Multi-Agent Reinforcement Learning (MARL) methods focus on this heterogeneous setting. When agents are heterogeneous, widely used techniques such as parameter sharing become detrimental to the learning of optimal policies. By using parameter sharing, agents effectively learn a shared policy, which limits their ability to behave differently. MARL solutions that try to effectively solve heterogeneous multi-agent systems therefore suffer on the scalability of their method, rendering them ineffective for large-scale settings. In this thesis the HCL framework is introduced, which aims to solve the two-sided problem of ensuring diverse agent behaviour and scalable learning for heterogeneous MARL. HCL overcomes the limitations that plague many MARL methods in heterogeneous multi-agent systems by learning distinct representations of environment observations for different agent types through contrastive learning. Because the learning of these representations is decoupled from MARL, HCL is able to use parameter sharing without suffering on diversity in agent behaviour. Through an experimental analysis on two heterogeneous multi-agent systems, we show that the use of distinct representations per agent type enhances the quality of the learned agent behaviour. Additionally, our results show that representation learning can be applied in novel ways to improve the performance of MARL compared to existing applications.
Collections
Related items
Showing items related by title, author, creator and subject.
-
'To learn or not to learn?' A study about the contribution of learning processes to national policy trajectories
Brandhorst, M. (2018)The growing network-based nature of society and increasing ‘wickedness’ of societal challenges incites the need for learning. Especially, in the field of spatial planning, since the desire for collaborative understanding ... -
Learning Strategies to Aid L2 English Vocabulary Retention: Classroom Learning Compared to E-Learning Using Words&Birds
Hoorn, G.T. van den (2017)The purpose of this pioneering exploratory study is to gain insight into the vocabulary learning strategies applied by young children (aged 9-12) in Dutch primary education involved in EarlyBird schools offering Early ...