Get More Code Reviews With This Simple Tool

Are you tired of prodding fellow developers to review your Pull Request? Then the code-review-leaderboard NPM package might be for you. You can show recognition to the devs who perform the most code reviews, and you can expose the slackers!

What does it do?

For a given repository-hosting platform (e.g. GitLab), all PR data is scraped over a given date range. This data is then processed to generate a leaderboard of users based upon their Code Reviewing activity. A user’s position on the board is calculated according to the following criteria:

• Number of PR approvals given
• Number of PR comments given
• Number of PR s created

Demo

In this demo, I will generate a leaderboard for the code reviews performed at my organisation between May 6th 2021 and May 9th 2021.

Some of our repositories are hosted in GitLab, while others are hosted in Azure, so I select both organisations.

I then enter the base URL for my Azure organisation, which has the form https://dev.azure.com/MyOrg/.

To authenticate the API requests to Azure, I must also enter a Personal Access Token. If you are unsure of how to generate a Personal Access Token, you can view the instructions for Azure or GitLab.

Finally I enter the base URL for my GitLab organisation, which has the form https://gitlab.example.com/ and then enter my GitLab Personal Access Token.

How does it work?

The steps for gathering Pull Request data differ depending on the API you are interacting with. These are the steps for Azure:

Step 1 – Determine the repository names

Endpoint – https://dev.azure.com/{organization}/_apis/git/repositories

This endpoint returns a list containing data for each repository in your organisation. From this information, we can extract a list of repository names. This will be important later, since it will be used to build future queries.

export const fetchAzureRepositoryData = async (): Promise<AzureRepository[]> => {
    let repositoryLookupResponse: GaxiosResponse<AzureRepositoryResponse> | undefined;

    await request<AzureRepositoryResponse>({
        baseUrl: getConfig().azure.baseUrl,
        url: `/_apis/git/repositories`,
        method: "GET",
        headers: getAzureHttpHeaders(),
        timeout: getConfig().httpTimeoutInMS,
        retry: true,
    })
        .then((response: GaxiosResponse<AzureRepositoryResponse>) => {
            validateSuccessResponse(response);
            repositoryLookupResponse = response;
        })
        .catch((response: GaxiosError<AzureRepositoryResponse>) => handleErrorResponse(response));

    return repositoryLookupResponse?.data.value ?? [];
};

Step 2 – Find the relevant Pull Request IDs

Endpoint – https://dev.azure.com/{organization}/{project}/_apis/git/pullrequests

This returns a list containing data for each Pull Request in a given repository. By coupling this with the repository names gathered in step 1, we now have the ability to retrieve data for all Pull Requests in your organisation.

export const fetchAzurePullRequestsByProject = async (projectName: string): Promise<AzurePullRequest[]> => {
    let pullRequestLookupResponse: GaxiosResponse<AzurePullRequestResponse> | undefined;

    await request<AzurePullRequestResponse>({
        baseUrl: getConfig().azure.baseUrl,
        url: `/${projectName}/_apis/git/pullrequests`,
        method: "GET",
        params: getAzureHttpParams(),
        headers: getAzureHttpHeaders(),
        timeout: getConfig().httpTimeoutInMS,
        retry: true,
    })
        .then((response: GaxiosResponse<AzurePullRequestResponse>) => {
            validateSuccessResponse(response);
            pullRequestLookupResponse = response;
        })
        .catch((response: GaxiosError<AzurePullRequestResponse>) => handleErrorResponse(response));

    return pullRequestLookupResponse?.data.value ?? [];
};

We then filter out any Pull Requests that were not modified within the given date range, then extract the IDs of the remaining Pull Requests.

Step 3 – Fetch the Pull Request activity

Endpoint – https://dev.azure.com/{organization}/_apis/git/repositories/{repositoryId}/pullRequests/{pullRequestId}/threads

This endpoint depends on the data obtained in the previous 2 steps.

• {project} – the repository name
• {repositoryId} – also the repository name (or the repository number)
• {pullRequestId} – the pull request id

export const fetchPullRequestNotes = async (projectName: string, pullRequestID: number): Promise<AzurePullRequestNote[]> => {
    let pullRequestLookupResponse: GaxiosResponse<AzurePullRequestNoteResponse> | undefined;

    await request<AzurePullRequestNoteResponse>({
        baseUrl: getConfig().azure.baseUrl,
        url: `/${projectName}/_apis/git/repositories/${projectName}/pullrequests/${pullRequestID}/threads`,
        method: "GET",
        headers: getAzureHttpHeaders(),
        timeout: getConfig().httpTimeoutInMS,
        retry: true,
    })
        .then((response: GaxiosResponse<AzurePullRequestNoteResponse>) => {
            validateSuccessResponse(response);
            pullRequestLookupResponse = response;
        })
        .catch((response: GaxiosError<AzurePullRequestNoteResponse>) => handleErrorResponse(response));

    return pullRequestLookupResponse?.data.value ?? [];
};

The data returned from this endpoint is referred to as threads. This includes a list of all activities that occurred on a pull request, including:

• Comments
• Approvals
• Commits
• Mentions
• System messages

We then filter out any threads which have not been modified within the given date range.

Step 4 – Parse Pull Request note data

Unfortunately the response does not explicitly state whether the thread item is a comment or an approval. But we can determine that using these key indicators:

• Only comments will have noteData.commentType set to "text"
• Only approvals will have noteData.content set to "<name> voted <number>"

const determineNoteType = (data: AzureComment): NoteType => {
    if (data.commentType === "text") {
        return NoteType.Comment;
    } else if (data.content.match(/[a-zA-Z ]+ voted [0-9]+/)) {
        return NoteType.Approval;
    } else {
        return NoteType.Unknown;
    }
};

Step 5 – Calculate the results

Finally, we can start generating our leaderboard. To make calculations easier, all response data has been parsed into a list of PullRequest Objects. Then for each user, we iterate through the list of Pull Requests to determine how many comments and approvals they gave.

export const calculateResults = (pullRequests: PullRequest[]): Result[] => {
    const uniqueNames: string[] = [...new Set(pullRequests.map((pullRequest) => pullRequest.authorName))];
    const results: Result[] = uniqueNames.map((name: string) => new Result(name));

    for (const result of results) {
        for (const pullRequest of pullRequests) {
            if (pullRequest.authorName === result.name) {
                result.numPullRequests++;
            }

            for (const note of pullRequest.notes) {
                if (note.authorName === result.name && note.noteType === NoteType.Approval) {
                    result.numApprovals++;
                } else if (note.authorName === result.name && note.noteType === NoteType.Comment) {
                    result.numComments++;
                }
            }
        }
    }

    return results;
};

Step 6 – Sort the results

The results are sorted based on the following priorities:

1. Number of PR approvals given
2. Number of PR comments given
3. Number of PRs created
4. Alphabetical naming

export const sortResults = (results: Result[]): Result[] => {
    return results.sort(
        (result1: Result, result2: Result) =>
            result2.numApprovals - result1.numApprovals ||
            result2.numComments - result1.numComments ||
            result2.numPullRequests - result1.numPullRequests ||
            result1.name.localeCompare(result2.name),
    );
};

Step 7 – Log the results

By using this neat little NPM package table, we can easily generate a nice looking table of results to the command line.

export const createResultsTable = (results: Result[]): (number | string)[][] => {
    const tableResults: (number | string)[][] = [];
    tableResults.push(TABLE_HEADINGS);

    results.forEach((result: Result) => {
        tableResults.push([result.name, result.numPullRequests, result.numComments, result.numApprovals]);
    });

    return tableResults;
};

Why should I use this?

• Recognition – Reviewing code is often a thankless task. This way there is increased visibility of those who donate a lot of time to reviewing their peer’s code.
• Visibility – If a developer is aware that their code reviewing efforts are going to be seen by others, then they may be more likely to dedicate time to it. No one wants to be at the bottom of the leaderboard.

Why shouldn’t I use this?

• Fallible – A dev can easily manipulate the leaderboard results by spamming comments on a PR, or approving already-merged PRs.
• Tension – If a dev is exposed to be performing little-to-no code reviews, they may receive criticism from their peers.
• Unbalanced– A dev cannot approve their own PR. This means that devs who create many PRs will have a lower number of potential comments and approvals.
• Misleading – The amount of effort one puts into reviewing code is not directly proportional to the amount of comments and approvals they have given. For example, reviewing a single 40-file change PR would take longer than reviewing multiple 1-row change PRs.

What are the limitations?

Currently code-review-leaderboard supports only 3 platforms: Github, GitLab, and Azure Repos. If you would like more platforms to be added, then feel free to raise a feature request here.

UPDATE(03/05/22): Github is now also supported