Detecting, understanding, and resolving build and test conflicts

Abstract

During collaborative software development, developers usually adopt branching and merging practices when making their contributions, allowing them to contribute to a software project independently. Despite such benefits, branching and merging come at a cost — the need to merge software and to resolve merge conflicts, which often occur in practice. While modern merge techniques, such as 3-way or structured merge, can automatically resolve many such conflicts, they fail when conflicts arise at the semantic level, known as semantic conflicts. These conflicts are revealed by failures when building and testing integrated code, build and test conflicts, respectively. Detecting such semantic conflicts requires understanding the software’s behavior, which is beyond the capabilities of most existing merge and assistive tools. To address the need for better assistive tools, we investigate semantic conflict occurrence by finding their causes and proposing tools that could support developers when facing these conflicts during merge scenario integrations. Initially, we perform a study investigating the frequency, structure, and adopted resolution patterns of build conflicts by empirically analyzing 451 open-source Java projects. As a result, we provide a catalogue of conflicts with 239 occurrences spread into six categories. Most build conflicts are caused by missing declarations removed or renamed by one developer but referenced by someone else. Further, analyzing some of these conflicts, we also report a catalogue of resolution patterns. In order to evaluate the occurrence of test conflicts, we adopt a different approach as these conflicts involve the semantics of a program. Consequently, they can not be detected during the compilation phase of the build process. This way, initially, we perform a second study investigating their occurrence by exploring the automated creation of unit tests as partial specifications to detect conflicts. Relying on a ground-truth dataset of more than 80 mutually integrated changes’ pairs on class elements from 51 software merge scenarios, we manually analyzed them and investigated whether test conflicts exist. Next, we systematically explore the detection of conflicts through four unit-test generation tools, as also the adoption of Testability Transformations aiming to increase the testability of the code under analysis. As a result, we provide a catalogue of 28 test conflicts, of which 9 of them were detected by our approach. Our results show that the best approach to detect conflicts involves combining the tools Differential EvoSuite and EvoSuite applied with Testability Transformations. As a final contribution, we present SAM, a semantic merge tool based on unit test generation, which warns developers about prominent conflicts on ongoing merge scenario integration. Overall, our results bridge a gap in the literature regarding the occurrence of code integration semantic conflicts during software development. Based on each conflict type, we investigate their causes and options to deal with them. While build conflicts could be automatically detected and fixed by an automatic repair tool, test conflicts might be detected by a semantic merge tool based on unit test generation.