Byte Size20min
Bug fixed, now what?
This proposal introduces an open-source GitHub bot that uses LLMs and search-based testing to automatically generate regression tests for bug fixes. Deployed in Mozilla’s Python repositories, the bot ensures bugs remain fixed and encourages adoption and feedback from the wider open-source community.
Konstantinos KitsiosUniversity of Zurich
Alberto BacchelliUniversity of Zurich
talkDetail.whenAndWhere
Friday, February 6, 12:40-13:00
Room C
When a bug is fixed, how can we make sure it will not be re-introduced in the future?
Of course, through a proper regression test, i.e., a test that fails in the presence of the bug and passes in its absence.
However, regression tests are frequently ommited by the developers, who tend to submit the patch alone.
How can AI automation help developers it this scenario?
We explore this question in depth by introducing our open-source GitHub bot that is triggered when a pull request is opened, and proposes a test that is guaranteed to fail in the presence of the bug and pass in its absence.
We use a combination of LLMs and traditional, search-based testing techniques to achieve this goal.
The bot is currently deployed in three Python repositories (Javascript and Rust will be soon also supported) of the Mozilla corporation, where it has already helped developers with their regression tests. We share interesting early insights and feedback from this deployment and invite more open-source developers to adopt the bot and provide constructive feedback!
Of course, through a proper regression test, i.e., a test that fails in the presence of the bug and passes in its absence.
However, regression tests are frequently ommited by the developers, who tend to submit the patch alone.
How can AI automation help developers it this scenario?
We explore this question in depth by introducing our open-source GitHub bot that is triggered when a pull request is opened, and proposes a test that is guaranteed to fail in the presence of the bug and pass in its absence.
We use a combination of LLMs and traditional, search-based testing techniques to achieve this goal.
The bot is currently deployed in three Python repositories (Javascript and Rust will be soon also supported) of the Mozilla corporation, where it has already helped developers with their regression tests. We share interesting early insights and feedback from this deployment and invite more open-source developers to adopt the bot and provide constructive feedback!
Alberto Bacchelli
Alberto Bacchelli is a professor of Empirical Software Engineering at University of Zurich,
Switzerland, where he leads the Zurich Empirical Software Engineering Team (ZEST).
His broader research vision is to innovate software engineering, through fundamental empirical
research and software tools. His goal is to increase our scientific knowledge of today’s software
development and to design, based on strong empirical evidence and theory, the right tools, languages, and development environments for high‑quality software engineering.
Switzerland, where he leads the Zurich Empirical Software Engineering Team (ZEST).
His broader research vision is to innovate software engineering, through fundamental empirical
research and software tools. His goal is to increase our scientific knowledge of today’s software
development and to design, based on strong empirical evidence and theory, the right tools, languages, and development environments for high‑quality software engineering.
comments.speakerNotEnabledComments