Skip to Main content Skip to Navigation
Journal articles

An approach and benchmark to detect behavioral changes of commits in continuous integration

Benjamin Danglot 1 Martin Monperrus 2 Walter Rudametkin 1 Benoit Baudry 2, 3
1 SPIRALS - Self-adaptation for distributed services and large software systems
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
3 DiverSe - Diversity-centric Software Engineering
Inria Rennes – Bretagne Atlantique , IRISA-D4 - LANGAGE ET GÉNIE LOGICIEL
Abstract : When a developer pushes a change to an application’s codebase, a good practice is to have a test case specifying this behavioral change. Thanks to continuous integration (CI), the test is run on subsequent commits to check that they do no introduce a regression for that behavior. In this paper, we propose an approach that detects behavioral changes in commits. As input, it takes a program, its test suite, and a commit. Its output is a set of test methods that capture the behavioral difference between the pre-commit and postcommit versions of the program. We call our approach DCI (Detecting behavioral changes in CI). It works by generating variations of the existing test cases through (i) assertion amplification and (ii) a search-based exploration of the input space. We evaluate our approach on a curated set of 60 commits from 6 open source Java projects. To our knowledge, this is the first ever curated dataset of real-world behavioral changes. Our evaluation shows that DCI is able to generate test methods that detect behavioral changes. Our approach is fully automated and can be integrated into current development processes. The main limitations are that it targets unit tests and works on a relatively small fraction of commits. More specifically, DCI works on commits that have a unit test that already executes the modified code. In practice, from our benchmark projects, we found 15.29% of commits to meet the conditions required by DCI.
Document type :
Journal articles
Complete list of metadata
Contributor : Lionel Seinturier Connect in order to contact the contributor
Submitted on : Tuesday, January 26, 2021 - 3:48:11 PM
Last modification on : Tuesday, January 4, 2022 - 6:50:49 AM

Links full text



Benjamin Danglot, Martin Monperrus, Walter Rudametkin, Benoit Baudry. An approach and benchmark to detect behavioral changes of commits in continuous integration. Empirical Software Engineering, Springer Verlag, 2020, 25 (4), pp.2379-2415. ⟨10.1007/s10664-019-09794-7⟩. ⟨hal-03121735⟩



Les métriques sont temporairement indisponibles