Skip to content
ChangeGuard

Code-change risk analysis, with provenance

Evidence for every risk in your diff.

ChangeGuard reads a change the way a careful reviewer does. It rebuilds both revisions, follows every caller, reruns static analysis on what is new, and measures the lines your tests never touch. A language model may explain the result, but only where the evidence holds.

Take the guided experienceAnalyse a diff

Python, JavaScript and TypeScript in depth; secrets, migrations, and dependencies for any file. Runs locally, needs no API key, and never executes the code it reads.

src/billing/pricing.pySAMPLE · SYNTHETIC
10−if percent > 50:
11+if percent >= 50:
15−def format_price(amount: Decimal, currency: str = "USD") -> str:
16+def format_price(amount: Decimal, locale: str) -> str:
9 stages · Ruff, tree-sitter, coverage30 ms
  • ◆2 call site(s) incompatible with new signature of format_pricesrc/billing/pricing.py:16High
  • ◆Undefined name REGIONAL_RATESsrc/billing/pricing.py:27High
  • ◆No test executes src/billing/api.py (3 changed executable line(s))src/billing/api.py:4Medium
  • ◆Async functions should not call time.sleepsrc/billing/api.py:11Medium
+9 more findings in the full report
Fig. 1A replay of a real run on the billing sample (synthetic data): a refactor that drops a parameter callers still pass. 13 findings in 30 ms. Walk through it.

A finding is only as good as the way it was found.

Most review tools merge everything into one confident list. ChangeGuard keeps three kinds of knowledge apart, labels each finding with its kind, and never lets a weaker kind overrule a stronger one.

  1. Deterministic

    established by analysis

    A parse, a resolved call, a measured line. If the rule says a caller passes a keyword the new signature rejects, it found that call and checked it the way Python binds arguments.

    2 call site(s) incompatible with new signature of format_price

    × passes unexpected keyword argument currency

    src/billing/pricing.py:16E2E3E4

  2. Heuristic

    inferred by a rule

    A pattern worth a reviewer’s attention whose impact depends on context: a flipped comparison, a function no test references. Stated as an inference, with its confidence, never as a fact.

    Condition logic changed: > → >=

    Inputs exactly at the boundary (or on the side the condition now excludes/includes) take the other branch than before — the classic off-by-one / inverted-condition regression.

    src/billing/pricing.py:11E10

  3. AI-generated

    proposed by a model, then verified

    Explanations and extra risks from a language model, kept only if every evidence ID, file, line, and symbol they mention checks out. Shown beside the rule’s analysis, never in place of it.

    “The new signature of format_price changes the required parameter from currency to locale, making it incompatible with existing call sites.”

    llama3.2:3b · local, via Ollama · citesE4

Eleven stages, all of them visible.

No single score hides the work. Every report records what each stage did, how long it took, and what it skipped and why. On the billing sample the full deterministic pipeline took 30 ms.

Read the system card

  1. 01Parse inputs

    0.68 ms

    Validates and parses the patch, snapshot archive, and coverage report.

    Never extracts archives to disk or follows symlinks.

  2. 02Reconstruct revisions

    0.08 ms

    Applies the patch to the snapshot in memory to rebuild the base and head revisions.

    Never runs git hooks or repository scripts.

  3. 03Structural diff

    1.6 ms

    Parses both revisions with tree-sitter and diffs functions, classes, and signatures.

  4. 04Cross-file references

    1.0 ms

    Resolves imports and finds every caller of a changed symbol, then checks each call against the new signature.

  5. 05Static analysis

    24 ms

    Runs Ruff on both revisions and keeps only diagnostics the change introduced.

    Never executes the analysed code.

  6. 06Risk rules

    1.5 ms

    Secrets, injection sinks, migrations, dependencies, condition flips, and test-integrity rules on added lines.

  7. 07Test mapping

    0.38 ms

    Finds tests that import and exercise each changed symbol.

  8. 08Coverage

    0.42 ms

    Measures which changed lines the uploaded coverage report says never ran.

  9. 09AI synthesis

    off

    Optional: a model explains findings and proposes risks, citing evidence IDs only.

    Never sees secrets or prompt-injection text.

  10. 10Grounding verification

    off

    Rejects any model claim whose evidence, paths, lines, or numbers do not check out.

  11. 11Assemble report

    0.45 ms

    De-duplicates findings, verifies every evidence reference, and computes the review priority.

Models may explain. They may not invent.

The model sees only fenced evidence, with secrets masked and prompt-injection text withheld. It must cite evidence IDs for every claim. Its output then passes through a verifier that rejects the claim outright, never repairing it, when any check fails.

Architecture does the rest: model output can add an explanation or a capped, separately labelled risk. It cannot delete, downgrade, or rewrite what the rules found.

  • Evidence existsEvery cited ID must be one ChangeGuard produced for this change.
  • Paths are realFile paths mentioned must belong to the change or its evidence.
  • Lines are in rangeLine numbers must fall within the evidence the claim cites.
  • Symbols are attributedA function named in a claim must appear in the evidence it cites.
  • No invented numbersPercentages must appear in the evidence; coverage is never estimated.
  • No test resultsChangeGuard runs no tests, so claims that tests passed or failed are rejected.

Measured on labelled changes, including the ones it misses.

67 code changes with 65 hand-written labels and 9 negative controls, compared against two naive baselines. Two holdout sets were written only after the rules were frozen, and each was run once before anything changed.

Clean holdout results: first run of each set

SetPrecisionRecallFalse alarms on safe changes
Holdout · 21 cases83.3%75.0%67%
Holdout v2 · 13 cases100.0%81.8%0%

Small and synthetic: these numbers show consistency with the labelling guide, not real-world effectiveness.

Full evaluation

Precision

95% CI
  • ChangeGuard100.0%
  • ChangeGuard, diff only100.0%
  • Keyword search baseline59.3%
  • Flag-every-change baseline31.4%

Recall

95% CI
  • ChangeGuard93.8%
  • ChangeGuard, diff only72.3%
  • Keyword search baseline49.2%
  • Flag-every-change baseline16.9%
Fig. 2Precision and recall on all 67 cases, with 95% bootstrap intervals over cases (2,000 resamples). The diff-only row is the same engine without a repository snapshot. Evidence integrity: 0 of 102 findings cited evidence that failed an independent re-check.

Six things ChangeGuard will never do.

  • Run your code.Uploads are parsed, never executed. Git is invoked with hooks and external diff drivers disabled.
  • Send a secret to a model.Detected credentials are masked before any report, export, or prompt is built.
  • Let a model overrule a rule.AI output is additive and capped at medium confidence. It cannot remove a finding.
  • Dress a heuristic as a fact.Every finding carries its provenance, and severity is kept separate from confidence.
  • Invent a probability.No blended risk score. The review priority is a transparent rule over labelled findings.
  • Keep your repository.Archives are read in memory and discarded; reports store only the excerpts they cite.

Run it on your next pull request.

Start the engine and the web app with one command, or use the CLI as a CI gate that emits SARIF for code scanning. Local models via Ollama are optional; nothing requires a paid API.

Analyse a diffSee how it works
Local, with Docker
docker compose up# web: http://localhost:3000   engine: http://localhost:8000/api/docs
As a CI gate
changeguard analyze --git-repo . --base origin/main \  --format sarif --output changeguard.sarif --fail-on high