How to add comments to Jupyter Notebook rich diffs on GitHub

Problem

GitHub recently launched a new feature that renders Jupyter Notebook rich diffs on commits and pull requests (yay!).

If you use GitHub for Jupyter Notebook version control, you know how tricky and frustrating notebook JSON diffs can be.

GitHub’s new rich diff feature is a step in the right direction but collaboration is essential to our work in Jupyter Notebooks, and this is where GitHub’s new feature falls short: Your teammates can’t comment on the rich notebook diffs in pull requests or commits.

Collaboration depends on being able to review notebook changes and discuss them in the context of a specific notebook cell. To comment on a notebook diff from the GitHub UI, you’ll need to switch to the JSON version of commits or pull requests. But the JSON diff format has its own problems:

• You can’t leave comments at a notebook cell level, you can only write comments on an individual line level.
• Rich output elements (plots, images) are shown in textual format, making it hard to review and comment on.
• Lots of unimportant changes are shown (metadata, execution counts).
• You can’t comment on unchanged notebook cells.

These issues make for a complicated and unintuitive code review workflow. So how can we resolve this and review notebook diffs on GitHub more effectively?

Solution #1: ReviewNB

ReviewNB is a GitHub-verified marketplace app built specifically for notebook code reviews on GitHub. With a simple GitHub integration, ReviewNB provides you with rich notebook diffs and commenting for commits and PRs.

ReviewNB shows you rich notebook diffs with the ability to comment on a notebook cell or line level. This means your team’s discussion and input can take place with a high-level overview while giving cell-specific details, all in the same intuitive UI.

And best of all, you can install ReviewNB as a GitHub marketplace app.

The benefits of using ReviewNB to comment on notebook diffs

• Comment directly on the rich notebook diff on any GitHub pull request.
• Start a discussion in any notebook, on any cell, not just in a pull request or commit. This is useful for reviewing a notebook with non-technical stakeholders.

Source

• Comment at cell level so that your team has the context they need to stay on the same page.

Source

• Segment discussions on changes around individual lines, cells, notebooks, and PRs so that your team can focus anywhere from cell-specific discussion to high-level PR concerns.

You can read more about how to do this here.

• Large notebook diffs won’t timeout on ReviewNB (while they would on GitHub).
• All PR comments are posted directly to the GitHub PR, so there’s no vendor lock-in.
• All interactive HTML/JS outputs (for example, Plotly, Bokeh, widgets) are rendered in ReviewNB but not in GitHub.

Workarounds

They’re a little awkward to implement, but here are some workarounds you can use to leave comments on notebook changes in GitHub.

Workaround #1: Review rich diff, comment on textual diff

If you don’t want to leave the GitHub UI, you’ll need to switch between the JSON format and the rich diff to make comments on changes.

Here’s how you can leave comments using this workaround:

1. To view your notebook’s rich diff in GitHub, click the Files changed tab of the pull request or commit.

   

2. To leave a comment, you’ll need to switch to the JSON format of the diffs. Switch to the JSON view by clicking the <> button at the top right of the rich diff view.

3. Find the changes you would like to make a comment on in the non-rendered version of your notebook and click the plus sign next to the line containing the change to add your comment.

   

4. When you click the Start a review button, GitHub will display a non-rendered version of the change to your notebook along with your comment in the “Discussions” section of the pull request.

   

Once comments have been made in a notebook, you’ll notice they can only be viewed in the JSON view and not the rich diff view.

This is an unintuitive process that requires you to jump back and forth between rich diff (to review the notebook) and textual diff (to write comments). Plus, you’re slowed down more by large rich diffs timing out and HTML/JS outputs not rendering on GitHub.

Workaround #2: Review locally in Jupyter, comment on GitHub

Alternatively, you can review notebook changes on your local machine and write comments in GitHub. Unfortunately, local environments don’t by default provide rich diff renders, so you’ll need an extra tool for that.

nbdime has tools for diffing and merging notebooks in your local environment. You can also configure Git to use the nbdime diff and merge tools when you do git diff or git merge with your notebook.

While ndbime allows you to see rich diff on commits or direct file names, it does not support rich diff on pull requests. It also does not allow you to comment on pull requests or any changes made.

To comment on changes you review locally with nbdime, you will need to switch back to the GitHub UI and leave comments in the plain text JSON diff there.

Summary

It’s useful to be able to view notebook rich diffs with GitHub’s new feature but frustrating that the feature doesn’t support commenting on changes in pull requests.

Although there are workarounds to comment on notebook rich diffs as we’ve discussed here, neither makes for a streamlined workflow, since you need to move back and forth between the rich diff view and the plain text diff to leave your comments. ReviewNB solves this problem by allowing you to comment on notebook rich diffs at both the single-line and cell level.

ReviewNB is used across thousands of GitHub repositories and it’s easy to see why:

• Rich Jupyter notebook diffs for easy code review.
• Conduct context-specific discussions without leaving the notebook.
• Supports large notebook diffs (no timeouts).
• Renders rich HTML/JS outputs.
• All this in an intuitive, purpose-built UI.