Project index

File 04 / 06

Insight.

AI-assisted code review that catches the logic bugs static tools miss.

Year

2023

Role

ML Engineer & Product

Timeline

5 months

Status

Internal tool

Stack

PythonPyTorchHugging FacePostgreSQL

01 — Overview

Insight reads every pull request at Intactic and reviews it like a senior engineer who has infinitely many coffee refills — trained on six years of our own review history, tuned to our conventions and our past mistakes.

02 — The challenge

Generic LLM review noise trained on the whole internet buried real findings under style nitpicks. Static analyzers caught the bugs we were never going to write anyway. The gap — actual logic errors, race conditions, broken invariants — stayed wide open.

03 — Approach

  • Fine-tuned on 14,000 of our own resolved review threads, so the model learned our invariants, not the internet’s
  • Built a retrieval layer over the codebase and incident history, grounding every comment in how the system actually behaves
  • Ranked findings by blast radius and confidence, and suppressed anything below the bar — silence is a feature
  • Measured against a hidden set of past real bugs: the ones that reached production before Insight existed

04 — The outcome

Insight now catches roughly one would-be-production bug per week that static tooling never flagged. Reviewer focus shifted from typo-hunting to architecture — which was always the point.

68%Real-bug recall
1/wkBugs caught pre-prod
14KTraining reviews

Next file — 05

Relay

Team messaging