Can an AI vendor credibly review its own code?

Partially. A separate review pass catches real issues even when it runs on the same model - a fresh context without the generation history reads the diff differently. What it cannot credibly do is catch the model's own systematic blind spots: research shows LLM evaluators recognize and favor their own generations, and prefer output that feels familiar to them. Credibility rises with distance - different pass, different model, different vendor, different reference.

Does Cursor Bugbot review code that Cursor itself wrote?

Often, yes - that is the workflow it is built for: the agent writes, Bugbot reviews the PR. Bugbot is a capable reviewer, and practitioners report it catching real bugs, including in PRs written by other vendors' agents. The independence question is not about Bugbot's quality; it is about how much distance exists between generator and reviewer when both are frontier LLMs, possibly the same one under the hood.

Is using a different model for review enough independence?

It is a real improvement - a differently trained model has different defaults and flags things the author-model considers normal. But cross-model review still shares one structural limit: the diff remains the only reference. If the change is well-built but does the wrong thing, no reviewer sees it, because the task is not in the diff. Full independence needs a second dimension: checking against the written task, not just the code.

What does research actually show about self-preference bias?

Two findings matter here. Panickssery, Bowman and Feng (2024) showed LLM evaluators recognize their own generations with non-trivial accuracy, and that self-recognition correlates linearly with self-preference. Wataoka, Takahashi and Ri (2024) located the mechanism: models rate familiar (low-perplexity) text higher - GPT-4 showed significant self-preference. Applied to code: the patterns a model always produces look correct to that model, by construction.

So is AI self-review useless?

No - that would be the opposite overcorrection. A same-model review pass in a fresh context catches real defects and is much better than no machine pass at all. The honest framing is residual risk: self-review reduces random errors but not systematic ones. Teams that understand this run self-review for speed and add independent checks - different model, deterministic tools, spec comparison - where correctness matters.

How does a team add independence without buying another platform?

Three steps that cost little: run review with a different model than generation (many tools let you pick); put deterministic checks - types, tests, linters - in front, since they have no model bias at all; and write the task down before the run, so verification has a reference the generating model cannot have shaped. The last step is the cheapest and the most neglected.

Comparison

Why AI Self-Review Is Not Enough

Last updated: 2026-07-024 min read

Independent verification means the instance checking AI-generated code is not the instance that produced it – not the same session, ideally not the same model, and not the code as its own reference. The concern is measured, not hypothetical: LLM evaluators recognize and favor their own generations, so a model reviewing its own code checks its blind spots with the same blind spots.

Contents

The convenient default: the assistant reviews itself

The 2026 tool landscape makes self-review the path of least resistance. Claude Code writes a change and reviews it on request; Cursor’s agent writes a PR and Bugbot reviews it; Copilot generates code and Copilot code review comments on it. To be fair: these reviewers are good, and a second pass in a fresh context catches real defects even on the same model – practitioners report solid results from these pipelines, especially across vendors.

The question this page examines is narrower and harder: what happens to the errors the generating model produces systematically– the patterns it always writes, the assumptions it always makes? Those are exactly the findings a reviewer sharing the generator’s training will not flag, because to that reviewer they look normal.

What the research shows: models favor their own output

This is measured, not folklore. Panickssery, Bowman and Feng (2024) showed that LLM evaluators recognize their own generations with non-trivial accuracy – and that self-recognition capability correlates linearly with the strength of self-preference. The bias appears out of the box, without any fine-tuning toward it.

Wataoka, Takahashi and Ri (2024) located the mechanism: models rate low-perplexity text – text that feels familiar to them – higher than human judges do, and GPT-4 showed significant self-preference in their measurements. Applied to code review, the implication is uncomfortable: the constructs a model reaches for by default are, by definition, the most familiar to it. A practitioner put it plainly: same model twice means the same blind spots twice.

The independence ladder

Level	Setup	What it adds	What remains
0	Same session checks itself	Almost nothing - the context that made the error rereads it	Everything
1	Same model, fresh pass	Catches slips the generation context masked	All model-systematic patterns
2	Different model / vendor reviews	Different training, different defaults - flags the author-model's habits	The diff is still its own reference
3	Deterministic checks (types, tests, linters)	Zero model bias on the mechanical layer	Cannot judge intent or semantics
4	Verification against the written task	A reference the generator did not author - catches 'right code, wrong thing'	Needs the task written down first

Independence in AI code checking is a ladder, not a switch - each rung removes a class of shared blind spots, and the last rung changes the reference, not just the reviewer.

Most teams stop at level 1 because it is the default their tooling ships with. The jump that changes the game is not 1→2 but 2→4: as long as the diff is the only reference, even a perfectly independent reviewer can only ask “is this good code?” – never “is this the right change?”

What an independent setup looks like in practice

Write the task down before the run. Goal, boundaries, checkable criteria – this creates the one reference the generating model cannot have shaped.
Let deterministic checks go first. Types, tests, linters have no perplexity preference; they clear the mechanical layer bias-free.
Review with distance. A different model than the generator where your tooling allows it – the research says the difference is what buys detection.
Keep the merge decision human. Independence culminates in someone accountable reading the evidence and deciding – no reviewer, however unbiased, replaces that.

Where Reality Graph fits

Independence is Reality Graph’s design axis rather than a feature: it verifies each run against the written task – a reference the coding agent did not author – with validation separate from the generating session, and records the outcome in an evidence report. It works beside Claude Code, Cursor, Copilot and their built-in reviewers – it is the second, independent opinion, not a replacement for the first one.

Independent verification adds

A reference the generating model did not shape
Detection of author-model habits a same-model pass normalizes
Bias-free mechanical checks before any judgment call
Evidence a reviewer and an auditor can read

It does not claim

That self-review or Bugbot-style tools are useless - they catch real bugs
That a different model removes all bias - it removes shared bias
That any automated check replaces the human merge decision
That vendors act in bad faith - this is architecture, not ethics

If these boundaries fit how your team wants to ship:

Get early access See how it works

FAQ

Can an AI vendor credibly review its own code?: Partially. A separate review pass catches real issues even when it runs on the same model - a fresh context without the generation history reads the diff differently. What it cannot credibly do is catch the model's own systematic blind spots: research shows LLM evaluators recognize and favor their own generations, and prefer output that feels familiar to them. Credibility rises with distance - different pass, different model, different vendor, different reference.
Does Cursor Bugbot review code that Cursor itself wrote?: Often, yes - that is the workflow it is built for: the agent writes, Bugbot reviews the PR. Bugbot is a capable reviewer, and practitioners report it catching real bugs, including in PRs written by other vendors' agents. The independence question is not about Bugbot's quality; it is about how much distance exists between generator and reviewer when both are frontier LLMs, possibly the same one under the hood.
Is using a different model for review enough independence?: It is a real improvement - a differently trained model has different defaults and flags things the author-model considers normal. But cross-model review still shares one structural limit: the diff remains the only reference. If the change is well-built but does the wrong thing, no reviewer sees it, because the task is not in the diff. Full independence needs a second dimension: checking against the written task, not just the code.
What does research actually show about self-preference bias?: Two findings matter here. Panickssery, Bowman and Feng (2024) showed LLM evaluators recognize their own generations with non-trivial accuracy, and that self-recognition correlates linearly with self-preference. Wataoka, Takahashi and Ri (2024) located the mechanism: models rate familiar (low-perplexity) text higher - GPT-4 showed significant self-preference. Applied to code: the patterns a model always produces look correct to that model, by construction.
So is AI self-review useless?: No - that would be the opposite overcorrection. A same-model review pass in a fresh context catches real defects and is much better than no machine pass at all. The honest framing is residual risk: self-review reduces random errors but not systematic ones. Teams that understand this run self-review for speed and add independent checks - different model, deterministic tools, spec comparison - where correctness matters.
How does a team add independence without buying another platform?: Three steps that cost little: run review with a different model than generation (many tools let you pick); put deterministic checks - types, tests, linters - in front, since they have no model bias at all; and write the task down before the run, so verification has a reference the generating model cannot have shaped. The last step is the cheapest and the most neglected.

Keep reading

ComparisonGreptile vs. Local VerificationGreptile's codebase index is its honest strength and its biggest data question at once. Strengths, documented data path, VPC self-hosting, and when a local verification layer fits better.ComparisonSonarQube AI Code Assurance vs. VerificationStricter static quality gates for AI code - deterministic, on-prem capable - split the five AI failure classes with verification: the gate owns code patterns, verification owns task conformance.All articlesThe whole collection – 43 cited, dated guides on verifying AI-generated code.