Venue
International Joint Conference on Artificial Intelligence
Domain
artificial intelligence, natural language processing, multi-agent systems
Large Language Models (LLMs) have achieved remarkable success across a wide array of tasks.Due to the impressive planning and reasoning abilities of LLMs, they have been used as autonomous agents to do many tasks automatically.Recently, based on the development of using one LLM as a single planning or decision-making agent, LLMbased multi-agent systems have achieved considerable progress in complex problem-solving and world simulation.To provide the community with an overview of this dynamic field, we present this survey to offer an in-depth discussion on the essential aspects of multi-agent systems based on LLMs, as well as the challenges.Our goal is for readers to gain substantial insights on the following questions: What domains and environments do LLM-based multi-agents simulate?How are these agents profiled and how do they communicate?What mechanisms contribute to the growth of agents' capacities?For those interested in delving into this field of study, we also summarize the commonly used datasets or benchmarks for them to have convenient access.To keep researchers updated on the latest studies, we maintain an open-source GitHub repository, dedicated to outlining the research on LLMbased multi-agent systems.
This paper presents a comprehensive survey on Large Language Model based Multi-Agents (LLM-MA), discussing their progress, challenges, and applications across various fields. It explores the reasoning and planning abilities of LLMs that enable the development of multi-agent systems capable of complex problem-solving and world simulation. The survey addresses key questions regarding the domains and environments for LLM-based agents, their profiling, communication mechanisms, and how their capacities grow. It categorizes applications into problem-solving and world simulations, providing examples and discussing the critical need for specialized datasets and benchmarks to further research in this field. The paper also outlines significant challenges such as managing hallucinations, acquiring collective intelligence, and scaling up LLM-MA systems, while offering insights into future research opportunities.
This paper employs the following methods:
The following datasets were used in this research:
- HumanEval
- MBPP
- SoftwareDev
- RoCoBench
- Communicative Watch-And-Help (C-WAH)
- ThreeDWorld Multi-Agent Transport (TDW-MAT)
- HM3D v0.2
- MMLU
- MedQA
- GSM8K
- StrategyQA
- Chess Move Validity
- SOTOPIA
- Gender Discrimination
- Nuclear Energy
- Werewolf
- Welfare Diplomacy
- Chameleon
- Undercover
- Ultimatum Game TE
- Garden Path TE
- Wisdom of Crowds TE
- MovieLens-1M
- Review of progress in LLM-based multi-agent systems
- Categorization of applications into problem-solving and world simulation
- Identification of challenges in advancing LLM-based systems
The authors identified the following limitations:
- Scalability issues regarding resource requirements for multiple agents
- Challenges in evaluating collective agent behaviors
- Absence of comprehensive benchmarks in certain applications
- Number of GPUs: None specified
- GPU Type: None specified
Large Language Models
Multi-Agent Systems
AI
Social Simulation
Problem Solving
World Simulation
Benchmarks