Skip to yearly menu bar Skip to main content


Poster

Synthesizing Software Engineering Data in a Test-Driven Manner

Lei Zhang · Jiaxi Yang · Min Yang · Jian Yang · Mouxiang Chen · Jiajun Zhang · Zeyu Cui · Binyuan Hui · Junyang Lin

West Exhibition Hall B2-B3 #W-102
[ ] [ ] [ Project Page ]
Thu 17 Jul 4:30 p.m. PDT — 7 p.m. PDT

Abstract:

We introduce SWE-Flow, a novel data synthesis framework grounded in Test-Driven Development (TDD).Unlike existing software engineering data that rely on human-submitted issues, SWE-Flow automatically infers incremental development steps directly from unit tests, which inherently encapsulate high-level requirements.The core of SWE-Flow is the construction of a Runtime Dependency Graph (RDG), which precisely captures function interactions, enabling the generation of a structured, step-by-step development schedule.At each step, SWE-Flow produces a partial codebase, the corresponding unit tests, and the necessary code modifications, resulting in fully verifiable TDD tasks.With this approach, we generated 16,061 training instances and 2,020 test instances from real-world GitHub projects, creating the SWE-Flow-Eval benchmark.Our experiments show that fine-tuning open model on this dataset significantly improves performance in TDD-based coding.To facilitate further research, we release all code, datasets, models, and Docker images at Github.

Lay Summary:

Modern software development often involves writing tests to ensure code works correctly. Inspired by this practice, we created SWE-Flow, a new system that helps machines learn how to build software step by step by following test instructions. Unlike previous methods that depend on manually written issue reports, SWE-Flow learns directly from unit tests, which naturally reflect what the software is supposed to do. At each step, it provides a small piece of code, a test to check it, and suggestions on what needs to be changed—just like how a real developer would work. We used this approach to build a large dataset from real-world projects and showed that it helps AI models do a better job at writing software in a test-driven way. All our tools, data, and code are freely available at https://github.com/Hambaobao/SWE-Flow.

Chat is not available.