The manual overhead in constructing Multi-Agent Systems (MAS) has been a significant bottleneck. Developing applications that leverage AI agents to fulfill diverse user intents typically demands painstaking manual composition of plans, agent selection, and execution graph creation. This paper introduces a novel framework designed to automate these critical steps, paving the way for more efficient MAS development.
Orchestrating Intelligence: An Automated MAS Framework
The proposed framework replaces manual processes with a suite of software modules and an orchestrated workflow. Central to this system are an LLM-derived planner, natural language task descriptions, a dynamic call graph, an orchestrator that maps agents to tasks, and a sophisticated agent recommender. This recommender, a key innovation, employs a two-stage information retrieval (IR) system. It first utilizes a fast retriever and then an LLM-based re-ranker to identify the most suitable agents from both local and global registries. This automated multi-agent systems approach is further refined by experiments exploring embedder choices, re-ranker effectiveness, agent description enrichment, and the impact of a supervising critique agent.
Enhancing Recall and Robustness with Hierarchical Review
Experimental benchmarks demonstrate the efficacy of this automated multi-agent systems approach. The system notably outperforms the state-of-the-art in recall rate, exhibiting superior robustness and scalability. The inclusion of a critique agent, which holistically re-evaluates agent and tool recommendations against the overall plan, proved to be a critical enhancement. This comprehensive review and revision process further boosts the recall score, underscoring its essential role in building effective end-to-end multi-agent systems. The researchers observed a notable shift towards more accurate and reliable agent selection through this layered recommendation and critique mechanism, as detailed on arXiv.