Automating Code Reviews with AI: A Practical Integration Guide

By TCAL — Tech, Coding & AI Learning

Disclosure: This article may contain affiliate links. We only recommend products we believe in. See how we make money.

Human review is still the review that matters. Reviewers get tired, miss the same null-check twice, and argue about naming while a secret walks into main. An AI pass does not replace that conversation. It clears the floor so humans can spend the review on architecture and intent.

This page is the CI wiring guide. It sits next to the CI/CD pipeline design notes and the AI code-generation security checklist. For the editor you type in while the bot reviews the PR, see Cursor vs GitHub Copilot.

Desk note — who this is for / what it’s bad at: Teams who will comment a PR diff from CI and keep a human approval on main. Bad as a reason to buy a review SaaS when the Messages script is still unrun, and a poor substitute for tests.

What to check before you subscribe (or fail CI)

Listing identity: This page’s script is a Messages call plus a gh api comment. GitHub Copilot code review, CodeRabbit, and Greptile are hosted review products with their own prompts and seats. Branch protection is the human gate. A comment bot is not a required approver.

Before you fail CI on a bot or sign a review-SaaS quote:

  • Scope the diff, not the repo. Full-tree review is a hallucination source and a token bill. The prompt guide is the rubric; this job is the wire.
  • Skip the SaaS if you have not posted one comment from Actions with a key you already pay for. A vendor logo does not write blocking vs nit. Buy a product when the script’s noise is the problem, not the existence of a first pass.
  • Do not fail the build on nits. Style comments that block merge become the reason people skip CI. Blocking is secrets and clear security bugs — see the security checklist.
  • Do not buy a GPU so a local model can review PRs. A 4090 search does not make a 7B a better reviewer. If the diff cannot leave the building, that is local weights plus a self-hosted runner, not a better Cursor hop — the Cursor + Ollama page is the editor path, not this job.
  • Skip this page if you still have no pipeline. Wire CI/CD first. If the change is an agent with write tools, read building agents before you let a bot approve its own diffs. Retrieval over a handbook is vector databases, not a review seat.

Practical cadence: one structured prompt → one comment on the PR → fail only on blocking. Then decide whether a hosted reviewer is worth the seat. More articles live on the blog index. Amazon search links on this page use tcalnet-20; see how we make money.

What the bot is good at — and what it is not

Good at (pattern matching):

  • Classic bugs: nulls, off-by-ones, unchecked error returns
  • Secret-shaped strings and obvious injection (string-built SQL, raw HTML)
  • “This function has no test and no docstring”
  • “You changed the handler but not the types”

Bad at (judgment):

  • Whether this is the right product change
  • Whether the new table is the right table
  • Team taste (the 400-line PR vs. the five small ones)
  • Domain rules that are not in the diff (“refunds over $X need two keys”)

If the change is a legacy rewrite, the legacy refactoring guide is the better starting point than a drive-by bot comment.

Do not paste the patch into a curl string

An earlier version of this page stuffed $(cat diff.patch) into a JSON body. That breaks the moment the patch contains a quote, a backslash, or a */. Official Anthropic Messages clients encode the payload. Use them — the same claude-sonnet-4-20250514 model the Claude API tutorial already uses.

# scripts/ai_review.py
import json
import os
import sys
import anthropic

PROMPT = """You are reviewing a git diff for a production service.
Return ONLY JSON with this shape:
{"findings":[{"severity":"blocking"|"nit","file":"path","note":"..."}]}
Flag only: bugs, security issues, missing tests for new branches.
Do not rewrite the feature. Do not invent files that are not in the diff.
"""

def main() -> None:
    diff = sys.stdin.read()
    if not diff.strip():
        json.dump({"findings": []}, sys.stdout)
        return
    client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
    message = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=2048,
        system=PROMPT,
        messages=[{"role": "user", "content": diff[:200_000]}],
    )
    sys.stdout.write(message.content[0].text)

if __name__ == "__main__":
    main()

Cap the diff. A generated lockfile or a 6 MB snapshot will blow the context window and your token bill. .gitattributes linguist-generated and a paths-ignore on the workflow are cheaper than asking the model to “ignore package-lock.”

GitHub Actions: comment, then maybe fail

The job needs pull-requests: write to comment. It should not need contents: write. Store the API key as a repository or org secret — the same rule as every other CI secret.

name: AI Code Review
on:
  pull_request:
    types: [opened, synchronize]

permissions:
  contents: read
  pull-requests: write

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install anthropic
      - name: Review the PR diff
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GH_TOKEN: ${{ github.token }}
        run: |
          git diff origin/${{ github.base_ref }}...HEAD \
            -- . ':(exclude)package-lock.json' ':(exclude)dist/**' \
            > diff.patch
          python scripts/ai_review.py < diff.patch > review.json
          {
            echo "### AI review (first pass, not a human approval)"
            echo
            echo '```json'
            cat review.json
            echo '```'
          } > review.md
          gh api "repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments" \
            -f body="$(cat review.md)"
          python -c "
          import json, sys
          data = json.load(open('review.json'))
          blocking = [f for f in data.get('findings', []) if f.get('severity') == 'blocking']
          sys.exit(1 if blocking else 0)
          "

Parse failures happen. If the model returns prose instead of JSON, fail the parser, not the product — log the raw output and exit 0 on nits so a flaky bot does not become the reason people skip CI. The unit-testing guide is the place for actually proving behavior; this job is a lint with opinions.

Prompt rules that keep the comments useful

  1. Changed files only. Full-repo review is a vanity metric and a hallucination source.
  2. Severity in the contract. blocking vs nit is how you avoid failing the build on “consider renaming.”
  3. One concern per finding. A paragraph that mixes style and SQL injection gets ignored.
  4. Ask for absences. “New endpoint, no test file” is the comment humans forget.
  5. Log accept vs dismiss. If the team ignores the same note for a month, delete it from the prompt. The bot should get quieter, not louder.

A few prompt patterns help here: a system rubric, a JSON constraint, and a negative instruction (“do not invent files”). Do not ask the model to “be a principal engineer.” Ask it to fill a schema.

When not to turn this on

  • A one-person repo where you already read every line. The bot is noise.
  • A first draft that will be force-pushed in ten minutes. Review the stable PR.
  • Generated code you have already decided to treat as an artifact (protobufs, lockfiles).
  • Security-sensitive changes that need a named human reviewer. The bot can still comment; it cannot be the approval.

Branch protection still belongs on main — required checks, required human approval — as in the git workflow notes. The AI job is one check, not the gate.