Skip to yearly menu bar Skip to main content


Poster
in
Workshop: Multi-Agent Systems in the Era of Foundation Models: Opportunities, Challenges and Futures

General Modular Harness for LLM Agents in Multi-Turn Gaming Environments

Yuxuan Zhang · Haoyang Yu · Lanxiang Hu · Haojian Jin · Hao Zhang


Abstract:

We introduce a modular harness design for LLM agents that composes of perception, memory, and reasoning components, enabling a single LLM/VLM backbone to tackle a wide spectrum of multi-turn gaming environments without domain-specific engineering. Using classic and modern game suites as low-barrier, high-diversity testbeds, our framework provides a unified workflow for analyzing how each module affects performance across dynamic interactive settings. Extensive experiments demonstrate that the harness lifts gameplay performance consistently over un-harnessed baselines and reveals distinct contribution patterns—for example, memory dominates in long-horizon puzzles while perception is critical in vision-noisy arcades. These findings highlight the effectiveness of our modular harness design in advancing general-purpose agent, given the familiarity and ubiquity of games in everyday human experience.

Chat is not available.