Should we replace our merge tools?

Abstract

Merge conflicts often occur when developers concurrently change the same code artifacts. While state of practice unstructured merge tools (e.g git merge) try to automatically resolve merge conflicts based on textual similarity, semistructured and structured merge tools try to go further by exploiting the syntactic structure and semantics of the involved artifacts. Previous studies compare semistructured and structured merge with unstructured merge concerning the number of reported conflicts, showing, for most projects and merge situations, a reduction in favor of semistructured and structured merge. This evidence, however, might not be sufficient to justify industrial adoption of advanced merge strategies such as semistructured and structured merge. The problem is that previous studies do not investigate whether the observed reduction on the number of reported conflicts actually leads to integration effort reduction (Productivity) without negative impact on the correctness of the merging process (Quality). Besides, it is unknown how semistructured merge compares with structured merge. So, to decide whether we should replace our state of practice unstructured merge tools, we need to compare these merge strategies and understand their differences. We then first compare unstructured and semistructured merge. Our results and complementary analysis indicate that the number of false positives is significantly reduced when using semistructured merge when compared to unstructured merge. However, we find no evidence that semistructured merge leads to fewer false negatives. Driven by these findings, we implement an improved semistructured merge tool that further combines both approaches to reduce the false positives and false negatives of semistructured merge. Semistructured merge has shown significant advantages over unstructured merge, especially as implemented by our improved tool. However, before deciding to replace unstructured tools by semistructured merge, we need to investigate whether structured merge is a better alternative than semistructured merge. So, we compare semistructured and structured merge. Our results show that semistructured and structured merge differ on 24% of the scenarios with conflicts. Semistructured merge reports more false positives, whereas structured merge has more false negatives. Finally, we observe that adapting a semistructured merge tool to resolve a particular kind of conflict makes semistructured and structured merge even closer. Overall, our findings suggests that semistructured merge is a better replacement of unstructured tools for conservative developers, having significant gains with a closer behavior to unstructured tools than structured merge. Besides that, practitioners might be reluctant to adopt structured merge because of the observed performance overhead and its tendency to false negatives. So, when choosing between semistructured and structured merge, semistructured merge would be a better match for developers that are not overly concerned with semistructured extra false positives.