Towards better code reviews: using mutation testing to improve reviewer attention

Abstract

Code reviews, while effective, can be crippled by process smells if not performed correctly. A typical process smell that harms the efficacy of code reviews is the ‘Looks Good To Me’ (LGTM) smell, wherein a reviewer approves a code review task without reviewing the code attentively. Low-quality code reviews can be harmful, as they can cause bugs to slip into a product codebase leading to potentially severe consequences. In this paper, we propose an innovative solution to potentially minimize the occurrence of the LGTM smell commonly found in code reviews. We built a tool that is a proof-of-concept implementation of our solution, which incorporates the concept of mutation testing into code reviews. It provides a platform where pull request authors can apply mutations to the pull request code in GitHub. Reviewer attention and review efficacy are measured based on their mutation score. To the best of our knowledge, our proof of concept implementation is the first-ever code review tool that uses the concept of mutation testing. We validated our proposed solution with eight developers and received promising results.