AI Coding Evidence Reports
Last updated: 2026-07-024 min read
An AI coding evidence report records, per run: what was intended, what changed, what was validated and with what result, what was deliberately skipped, and what remains uncertain. Stored with the code, readable by the reviewer, the team, and the future.
Contents
Why 'it works' stopped being enough
When a person writes a change, the review conversation happens with someone who remembers why. When an AI wrote it, that memory does not exist. The prompt is gone, the reasoning is gone, and the summary was written by the same model whose work is in question. Sonar found 53% of developers have seen AI code that looks correct but isn’t reliable; “looks correct” is precisely the problem evidence exists to solve.
Evidence replaces recollection with a record. Not a heavyweight document - a short, structured answer to the five questions every reviewer silently asks.
The five questions a report answers
- Intent - what was this run supposed to do, in writing, from before it started?
- Change - what actually changed, and did it stay inside the declared boundaries?
- Validation - which checks ran (tests, types, lint, build, targeted checks), with what results?
- Negative space - what was deliberately not validated, and why? This is the part no self-written summary volunteers.
- Decision - who accepted it, knowing all of the above?
evidence-report.mdtext/markdown
Sample - illustrative, not real run dataRun: 2026-07-02 · fix-rate-limit-headers
Task: Return correct Retry-After header on 429 responses
Boundaries: api/middleware/* only · no changes to rate-limit logic
Tool: Claude Code
Changes: 2 files, +38 −7 (within boundaries ✓)
Validation
✓ unit tests (61 passing, 3 new - written before the run)
✓ type check, lint, build
✗ load test - SKIPPED: staging env unavailable
? header behavior behind CDN - UNCERTAIN, needs manual check
Decision: APPROVED by mk · load test to follow before releaseThe format matters less than the discipline: anchored to a pre-written task, honest about gaps, stored where the code lives - not in a chat scrollback.
From “it works” to an audit trail: four levels of proof
The same change can arrive with very different levels of proof. Only from the evidence report down does a reviewer get something they can check instead of trust - and the accumulated trail answers the general question a bare claim never touches.
| “It works” | A PR description | A per-run evidence report | An accumulated audit trail | |
|---|---|---|---|---|
| What it recordsThe information the artifact actually carries about the run | Nothing durable - a status in a chat window that scrolls away.Editorial reading · 2026-07 | A prose summary written after the fact, saying what its author chose to say.Editorial reading · 2026-07 | Intent, change, validation results, the negative space, and the decision - per run.Editorial reading · 2026-07 | The per-run reports over time, stored with the code they describe.Editorial reading · 2026-07 |
| Who authored itWhose account of the change this is | The agent, about its own work, in the same breath as the change.Editorial reading · 2026-07 | Usually the same model that made the change, unprompted about what it skipped.Editorial reading · 2026-07 | The verification workflow, from a task written before the run started.Editorial reading · 2026-07 | The team, as a byproduct - no one writes the trail up separately.Editorial reading · 2026-07 |
| What a reviewer can checkWhat a reviewer can verify from it without re-deriving the work | Nothing; the reviewer is back to reading the diff cold.Editorial reading · 2026-07 | The narrative, not the run; the negative space is whatever the summary omits.Editorial reading · 2026-07 | Which checks ran with what result, and - first-class - which ones did not.Editorial reading · 2026-07 | How the team controls AI-assisted change in general, not just one run.Editorial reading · 2026-07 |
| Where it falls shortThe failure mode that keeps this level from closing the trust gap | “Looks correct” is exactly the state that later turns out not to be reliable.Editorial reading · 2026-07 | No anchor to a pre-written task and no validation results, so nothing constrains it.Editorial reading · 2026-07 | Only as honest as the workflow feeding it; a skip must actually be recorded as a skip.Editorial reading · 2026-07 | It answers the general question, not the specific one; it supports an audit, it is not one.Editorial reading · 2026-07 |
What evidence changes in practice
- Reviews get faster and deeper at once - the reviewer starts from intent and validation status instead of reconstructing both from the diff.
- “Looked right, wasn’t” incidents become traceable- when something breaks, the report shows what was and wasn’t checked, turning blame into process improvement.
- An audit trail accumulates for free- per-run reports add up to a defensible answer to “how does your team control AI-assisted changes?” - a question engineering leads hear increasingly often.
- The team learns - skipped validations and recurring uncertainties are visible patterns, not anecdotes.
Where Reality Graph fits
Reality Graph generates evidence reports as a byproduct of its verification workflow. The task and validation plan are defined before the run, boundary checks and validation results are collected after it, and the report assembles itself - locally, stored with your code. How the single outcome on top of that report is derived, including the runs that stay blocked while every test is green, is documented under how the verdict is computed. It is in private beta; early access is open for a small group of teams.
What it does
- Generates the report from the verification workflow - no manual bookkeeping
- Records intent, boundaries, validation results, and open questions per run
- Keeps the negative space visible: skipped and uncertain items are first-class
- Stores reports locally, with your code - auditable by your team
What it does not do
- Invent evidence - what didn't run isn't reported as run
- Replace tests or CI - it records what they said, per run
- Ship your reports anywhere - they stay in your environment
- Claim compliance - evidence supports auditability; a certification it is not
FAQ
- What is an AI coding evidence report?
- A short, structured record attached to each AI coding run. What the task was, what changed, what validation ran and with what result, what was deliberately skipped, and what remains uncertain. It turns 'the agent said it's done' into something a reviewer can actually check.
- Isn't that just a pull request description?
- A PR description is written after the fact, usually by the same model that made the change, and says whatever its author chose to say. An evidence report is anchored to a task that was written down before the run and records validation results - including the negative space: what was not tested.
- Who reads evidence reports?
- Reviewers first - they get intent and validation status instead of a bare diff. Then future maintainers doing archaeology on a change, engineering leads who need to know how AI-assisted work is verified, and - in regulated contexts - auditors who ask how changes were controlled.
- Does writing evidence slow the team down?
- The report is generated from what the verification workflow already knows - the written task, the boundary checks, the validation results. The human cost is minutes of defining the task up front, which pays back at review time. Evidence that requires manual bookkeeping would indeed die; that is why it has to fall out of the workflow.
- What is the difference between evidence and an audit trail?
- Evidence is per run and answers 'was this change verified, and how?'. An audit trail is the accumulation over time and answers 'how does this team control AI-assisted changes in general?'. Reports stored with the code give you both without extra process.
- How long should a team keep evidence reports?
- As long as the code they describe - stored with the code, that takes care of itself, because the reports travel with the repository through branches, merges and archival. A report earns its keep exactly when someone asks, months later, what was actually checked on this change; reports filed away separately are, in practice, no longer findable by then.
Keep reading
GuideYour First Verification ReportA team lead can stand up a first AI-code verification report in an afternoon, no platform required - the tool-agnostic roadmap, a copyable report template, and the path from one report to a team practice.ComplianceAudit Trails for AI-Generated CodeGit records what changed - not what the task was, what was validated, who approved. The five auditor questions, why the demand rises under NIS2 and the new product liability, and how to build the trail.All articlesThe whole collection - 58 cited, dated guides on verifying AI-generated code.