CORRECT: Code Reviewer Recommendation Based on Cross-Project & Technology Experience

Abstract: Peer code review locates common coding rule violations and simple logical errors in the early phases of software development, and thus reduces overall cost. However, in GitHub, identifying an appropriate developer for code review during pull request submission is a non-trivial task. In this paper, we propose a heuristic ranking technique that considers not only the cross-project work history of a developer but also her experience in certain technologies associated with a pull request for determining her expertise as a potential code reviewer. We first motivate our technique using an exploratory study with 20 commercial projects. We then evaluate the technique using 13,081 pull requests from ten projects, and report 92.15% accuracy, 85.93% precision and 81.39% recall in code reviewer recommendation which outperforms the state-of-the-art technique.

CORRECT Plugin


  • Required Browser: Google Chrome
  • Supported OS: Any OS that supports Google Chrome.

Recommendation Enabled:
Currently, CORRECT's recommendation supports are enabled for the following software systems:

Commercial/Business Systems (10)

SystemTop-10 accuracy*
CS74.43%
ARM96.57%
SM93.14%
VW97.46%
MS90.71%
SR84.86%
NB95.10%
VBC82.71%
AA68.14%
ST89.42%
* Latest recorded Top-10 accuracy by experimenting with the most recent 700 pull requests from each subject system.

Internal Libraries (15):
  • VA-vapi
  • VA-vauth
  • VA-vautil
  • VA-vbackup
  • VA-vbootstrap
  • VA-vbuild
  • VA-vconsole
  • VA-vform
  • VA-vjira
  • VA-vlogs
  • VA-vmonitor
  • VA-vpipeline
  • VA-vpubsub
  • VA-vsandwich
  • VA-vtest

Related Publication(s)


@inproceedings{icse2016masud,
author = {Rahman, M. M. and Roy, C. K. and Collins, J.},
title = {{CORRECT: Code Reviewer Recommendation Based on Cross-Project and Technology Experience}},
booktitle = {Proc. ICSE},
year = {2016},
pages = {222--231} }
@inproceedings{ase2016masud-correct,
author = {Rahman, M. M. and Roy, C. K. and Redl, J and Collins, J.},
title = {{CORRECT: Code Reviewer Recommendation at GitHub for Vendasta Technologies}},
booktitle = {Proc. ASE},
year = {2016},
pages = {6, to appear} }

©Masud Rahman, Computer Science, University of Saskatchewan, Canada.