As scaling large language models faces prohibitive costs, multi-agent systems emerge as a promising alternative, though challenged by static knowledge assumptions and coordination inefficiencies. We introduce Knowledge-Aware Bayesian Bandits (KABB), a novel framework that enhances multi-agent system coordination through semantic understanding and dynamic adaptation. The framework features three key innovations: a customized knowledge distance model for deep semantic understanding, a dual-adaptation mechanism for continuous expert optimization, and a knowledge-aware Thompson Sampling strategy for efficient expert selection. Extensive evaluation demonstrates KABB achieves an optimal cost-performance balance, maintaining high performance while keeping computational demands relatively low in multi-agent coordination.

Problem. Scaling large language models and static expert ensembles incurs prohibitive compute and monetary costs.Solution. We propose Knowledge-Aware Bayesian Bandits (KABB), which (1) computes a semantic distance to match tasks and experts, (2) continuously adapts expert skills, and (3) employs a Thompson Sampling strategy that dynamically routes each task to the best subset of experts.Impact. On benchmark suites, KABB achieves equal or better accuracy while cutting computational expense by up to 7× compared to fixed ensembles.