Skip to content
Reality Graph

Concept

AI Coding Evidence Reports

Last updated: 2026-10-195 min read

An AI coding evidence report records, per run: what was intended, what changed, what was validated and with what result, what was deliberately skipped, and what remains uncertain. Stored with the code, readable by the reviewer, the team, and the future.

DiffChecksScopeone ruleverifiedwith limitsblocked
Contents

Why 'it works' stopped being enough

When a person writes a change, the review conversation happens with someone who remembers why. When an AI wrote it, that memory does not exist. The prompt is gone, the reasoning is gone, and the summary was written by the same model whose work is in question. Sonar found 53% of developers have seen AI code that looks correct but isn’t reliable; “looks correct” is precisely the problem evidence exists to solve.

Evidence replaces recollection with a record. Not a heavyweight document - a short, structured answer to the five questions every reviewer silently asks.

The five questions a report answers

  • Intent - what was this run supposed to do, in writing, from before it started?
  • Change - what actually changed, and did it stay inside the declared boundaries?
  • Validation - which checks ran (tests, types, lint, build, targeted checks), with what results?
  • Negative space - what was deliberately not validated, and why? This is the part no self-written summary volunteers.
  • Decision - who accepted it, knowing all of the above?

evidence-report.mdtext/markdown

Sample - illustrative, not real run data
Run:        2026-07-02 · fix-rate-limit-headers
Task:       Return correct Retry-After header on 429 responses
Boundaries: api/middleware/* only · no changes to rate-limit logic
Tool:       Claude Code

Changes:    2 files, +38 −7   (within boundaries ✓)

Validation
  ✓ unit tests (61 passing, 3 new - written before the run)
  ✓ type check, lint, build
  ✗ load test           - SKIPPED: staging env unavailable
  ? header behavior behind CDN - UNCERTAIN, needs manual check

Decision:   APPROVED by mk · load test to follow before release
The five answers on one screen: intent and boundaries from before the run, the change and its checks, the negative space marked explicitly. Skipped and uncertain are first-class - and who accepted it knowing all of it. It is a fabricated example built to show the shape, not a record of any real run.This is an illustrative sample, never presented as real run data; the discipline it shows matters more than the exact format.

The format matters less than the discipline: anchored to a pre-written task, honest about gaps, stored where the code lives - not in a chat scrollback.

From “it works” to an audit trail: four levels of proof

The same change can arrive with very different levels of proof. Only from the evidence report down does a reviewer get something they can check instead of trust - and the accumulated trail answers the general question a bare claim never touches.

The same change can arrive with four very different levels of proof; only from the evidence report down does a reviewer get something they can check instead of trust. This is the page's own reading of the ladder, not a scoring model.
“It works”A PR descriptionA per-run evidence reportAn accumulated audit trail
What it recordsThe information the artifact actually carries about the runNothing durable - a status in a chat window that scrolls away.Editorial reading · 2026-07A prose summary written after the fact, saying what its author chose to say.Editorial reading · 2026-07Intent, change, validation results, the negative space, and the decision - per run.Editorial reading · 2026-07The per-run reports over time, stored with the code they describe.Editorial reading · 2026-07
Who authored itWhose account of the change this isThe agent, about its own work, in the same breath as the change.Editorial reading · 2026-07Usually the same model that made the change, unprompted about what it skipped.Editorial reading · 2026-07The verification workflow, from a task written before the run started.Editorial reading · 2026-07The team, as a byproduct - no one writes the trail up separately.Editorial reading · 2026-07
What a reviewer can checkWhat a reviewer can verify from it without re-deriving the workNothing; the reviewer is back to reading the diff cold.Editorial reading · 2026-07The narrative, not the run; the negative space is whatever the summary omits.Editorial reading · 2026-07Which checks ran with what result, and - first-class - which ones did not.Editorial reading · 2026-07How the team controls AI-assisted change in general, not just one run.Editorial reading · 2026-07
Where it falls shortThe failure mode that keeps this level from closing the trust gap“Looks correct” is exactly the state that later turns out not to be reliable.Editorial reading · 2026-07No anchor to a pre-written task and no validation results, so nothing constrains it.Editorial reading · 2026-07Only as honest as the workflow feeding it; a skip must actually be recorded as a skip.Editorial reading · 2026-07It answers the general question, not the specific one; it supports an audit, it is not one.Editorial reading · 2026-07
The same change can arrive with four very different levels of proof; only from the evidence report down does a reviewer get something they can check instead of trust. This is the page's own reading of the ladder, not a scoring model.

What evidence changes in practice

  • Reviews get faster and deeper at once - the reviewer starts from intent and validation status instead of reconstructing both from the diff.
  • “Looked right, wasn’t” incidents become traceable- when something breaks, the report shows what was and wasn’t checked, turning blame into process improvement.
  • An audit trail accumulates for free- per-run reports add up to a defensible answer to “how does your team control AI-assisted changes?” - a question engineering leads hear increasingly often.
  • The team learns - skipped validations and recurring uncertainties are visible patterns, not anecdotes.

Who else gets cited on evidence reports for AI code, and where Reality Graph sits

Ask an AI search engine how to prove that AI code was verified and a report format is rarely the answer. What gets cited is GitHub’s tutorial on reviewing AI-generated code, service offerings such as Reply’s AI-assisted code review, trade press covering studies on security flaws in AI code, and practitioner threads in developer forums. Each of them does something an evidence report does not.

SourceWhat it does betterWhat Reality Graph does beside it
Vendor docs (GitHub Copilot review tutorial)Shows step by step how review works inside its own tool, with screenshots and current settings.Is tool-neutral and records per run what was asked, changed, checked and skipped, whether the code came from Copilot, Claude Code or Cursor.
Review as a service (Reply)People plus a platform that take over the review and produce documents. If you want to buy capacity, it is there.Is software, not a service. It produces the report from the workflow and leaves review and decision to your team.
Trade pressPlaces studies on security flaws in AI code in context and carries the news value a report format does not have.Supplies no study and no figures, but a five-question format that gets answered per run.
Practitioner threads in developer forumsUnfiltered experience with AI review tools, failures included, which no vendor page tells the same way.Is a vendor page and says so. What a report changes day to day is something your team has to observe for itself.
Four kinds of source AI answers cite on proving AI code was verified: what each does better and what Reality Graph does beside it. Editorial placement as of 19 October 2026, not a product rating.

Reality Graph therefore sits beside the coding tool and before the pull request: it files task, boundaries, checks and open points per run as a report next to the code. If you want to set up a specific tool’s review, buy review capacity or cite a study, the sources above are the right place.

Where Reality Graph fits

Reality Graph generates evidence reports as a byproduct of its verification workflow. The task and validation plan are defined before the run, boundary checks and validation results are collected after it, and the report assembles itself - locally, stored with your code. How the single outcome on top of that report is derived, including the runs that stay blocked while every test is green, is documented under how the verdict is computed. It is in private beta; early access is open for a small group of teams.

What it does

  • Generates the report from the verification workflow - no manual bookkeeping
  • Records intent, boundaries, validation results, and open questions per run
  • Keeps the negative space visible: skipped and uncertain items are first-class
  • Stores reports locally, with your code - auditable by your team

What it does not do

  • Invent evidence - what didn't run isn't reported as run
  • Replace tests or CI - it records what they said, per run
  • Ship your reports anywhere - they stay in your environment
  • Claim compliance - evidence supports auditability; a certification it is not
It does not invent evidence and it is not a compliance credential: it records what actually ran, per run, and leaves the reports with your code.

FAQ

What is an AI coding evidence report?
A short, structured record attached to each AI coding run. What the task was, what changed, what validation ran and with what result, what was deliberately skipped, and what remains uncertain. It turns 'the agent said it's done' into something a reviewer can actually check.
Isn't that just a pull request description?
A PR description is written after the fact, usually by the same model that made the change, and says whatever its author chose to say. An evidence report is anchored to a task that was written down before the run and records validation results - including the negative space: what was not tested.
Who reads evidence reports?
Reviewers first - they get intent and validation status instead of a bare diff. Then future maintainers doing archaeology on a change, engineering leads who need to know how AI-assisted work is verified, and - in regulated contexts - auditors who ask how changes were controlled.
Does writing evidence slow the team down?
The report is generated from what the verification workflow already knows - the written task, the boundary checks, the validation results. The human cost is minutes of defining the task up front, which pays back at review time. Evidence that requires manual bookkeeping would indeed die; that is why it has to fall out of the workflow.
What is the difference between evidence and an audit trail?
Evidence is per run and answers 'was this change verified, and how?'. An audit trail is the accumulation over time and answers 'how does this team control AI-assisted changes in general?'. Reports stored with the code give you both without extra process.
How long should a team keep evidence reports?
As long as the code they describe - stored with the code, that takes care of itself, because the reports travel with the repository through branches, merges and archival. A report earns its keep exactly when someone asks, months later, what was actually checked on this change; reports filed away separately are, in practice, no longer findable by then.
What are the alternatives to Reality Graph for proving AI code was verified?
The review features of the coding tools themselves, such as Copilot code review with its GitHub tutorial, document findings straight in the pull request and are better integrated there. Review service providers such as Reply supply people and a platform when capacity is missing. A self-maintained PR template with fixed fields costs nothing and is often enough for a small team. Reality Graph produces the report from the workflow per run, tool-neutral and locally. It replaces none of these alternatives and is not an audit.

Keep reading

Sources

Want to see what your last agent run would have looked like?

Request access