[Day 16 of 30] When AI sounds right and is still wrong: force an evidence check
Use Evidence-Gate to separate confirmed facts from assumptions before you decide what to do next
The situation
You ask the assistant a thinking question, and it replies fast, polished, and plausible.
But later you notice the failure pattern:
the assistant sounded sure
your actual system has missing or stale facts
you end up guessing what the assistant based the answer on
This is not an “AI accuracy” problem. It is a workflow problem. Thinking work needs an evidence trail, not just a good-sounding paragraph.
Typical pains you feel:
Latest decision is not recorded. The assistant references “what we agreed,” but the agreement never got written into your projects, people notes, or decisions log.
Current status is stale. The assistant describes “where things are,” while your backlog or tasks view has not been updated.
Ownership is unclear. The assistant assigns next steps to someone, but the real owner lives somewhere else in your system, or nowhere at all.
The 1% upgrade
Add one tiny gate before any “final” answer or decision: Evidence-Gate.
Evidence-Gate forces the assistant to:
make claims only after checking your provided context
separate what is confirmed from what is unverified
convert every unverified gap into one precise next question or one system update you should do
So you keep speed, but remove the hidden guessing.
Apply this in your workflow
Ask your assistant to apply the Evidence-Gate structure directly to your request. Use the input and output shape below.
What to provide in one message:
your decision request
your constraints (time, risk, audience)
your context pack (facts with source labels)
optional: your draft
What you should get back:
decision summary
confirmed facts
unverified gaps
next context updates
confidence
Reusable skills and slash commands
If you want this to work every day, turn it into reusable skills once. A custom skill here means a reusable slash command you create in your own setup, so the same quality check runs in a consistent format every time.
A skill is reliable only when three things are explicit:
output schema
safety rules
acceptance tests
If one of these is missing, quality will drift.
Cursor setup
Open chat in your project.
Paste the Copy-ready setup request block from this post as a single message.
Ask the assistant to confirm the exact slash command names it created.
Run acceptance tests before real usage:
Test A (single decision): /evidence-gate must return all 5 sections in order.
Test B (missing data): /evidence-gate must not fabricate and must show unverified gaps.
Test C (multi-topic): /thinking-work-os must produce a decision list (decisions + evidence + open questions + confidence) and concrete next actions.
Test both on one real case using slash commands:
/evidence-gate
/thinking-work-os
Validation check (must pass):
Both skills are created as runnable chat commands.
You can invoke them directly in chat:
/evidence-gate
/thinking-work-os
Running each command returns the expected output sections (from tests A/B/C), not a generic note template.
Claude Code setup
Open chat in your project.
Paste the Copy-ready setup request block from this post as a single message.
Confirm both commands are available.
Run the same acceptance tests (A/B/C above).
Then run the same two real-case tests using slash commands:
/evidence-gate
/thinking-work-os
Validation check (must pass):
Both skills are created as runnable chat commands.
You can invoke them directly in chat:
/evidence-gate
/thinking-work-os
Running each command returns the expected output sections (from tests A/B/C), not a generic note template.
Copy-ready setup request
Send this to your assistant as a one-message setup:
Create two skills for my project and expose them as slash commands:
1) evidence-gate
Purpose: one-decision audit output with confirmed facts, unverified gaps, next context updates, confidence.
Non-negotiable output sections: Decision summary, Confirmed facts, Unverified gaps, Next context updates, Confidence.
Safety rule: never invent facts.
2) thinking-work-os
Purpose: weekly or cross-topic decision cycle using the same evidence logic for high-impact topics.
Non-negotiable output sections: Priority topics, Decision list (a structured list of decisions with evidence, open questions, and confidence), Action plan, Record updates, Tool stack adjustments.
Safety rule: no forced decisions when evidence is weak, use “defer with questions”.
After creation, show me exactly how to run both as slash commands.
Then run these acceptance tests:
- A) /evidence-gate returns all 5 sections in order.
- B) /evidence-gate with missing facts shows unverified gaps and no invented facts.
- C) /thinking-work-os returns a decision list (decisions + evidence + open questions + confidence) and concrete next actions.
Then run this implementation check:
- Confirm both slash commands are available in chat.
- Confirm both commands produce the expected output sections from tests A/B/C.
- Hard requirements:
- Output must be runnable skills exposed as slash commands.
- Notes, templates, or document-only artifacts are invalid outputs.
- Keep required output sections and safety rules exactly as specified above.
- Definition of done:
- Slash commands exist and are discoverable.
- Both commands execute successfully.
- Acceptance tests A/B/C pass.
- Failure policy:
- If output is not runnable slash commands, treat as failed and regenerate before responding.
Which one to run
Use /evidence-gate when you need one fact-checked answer.
Use /thinking-work-os when you need a full decision cycle across multiple topics.
Example 1: Product decision with missing “why”
Decision request: “Should we ship Feature A to enterprise customers this week?”
Context Pack contains:
Project status says “Feature A in review”
No decision log about “enterprise readiness criteria”
No meeting note about compliance blockers
Assistant output after Evidence-Gate:
A) Decision summary
I can recommend “delay for now” only if we confirm enterprise readiness criteria. Right now, I know the feature is in review, but I do not have recorded evidence for enterprise-specific blockers or acceptance criteria.
B) Confirmed facts
Feature A is in review (Source: Projects)
There is no recorded enterprise readiness decision in my context (Source: Decisions log)
C) Unverified gaps
What enterprise-specific requirement must be satisfied to ship this week?
Is there a known compliance or security blocker for enterprise accounts?
D) Next context updates
Add a short entry to Decisions log: “Enterprise readiness criteria for Feature A”
Capture the blocker status in Projects: “Compliance/security check result and date”
E) Confidence
Medium
This is the difference you want: a decision that comes with a checklist of proof.
Example 2: Hiring screen with invented ownership
Decision request: “Is this candidate a strong fit for product discovery work?”
Context Pack contains:
Resume highlights
One People note about “previous role,” but no link to specific discovery outcomes
No recorded evidence of research methods used by the candidate
Evidence-Gate prevents the assistant from turning “sounds like” into “is.”
Evidence-Gate output would produce:
confirmed: what is explicitly in the provided notes
unverified: what you need to ask in the next interview round
next updates: where to record candidate evidence after the call
10-minute action
Pick one open thinking task you care about today (product decision, planning question, stakeholder update, hiring call).
Do this:
Generate the assistant’s recommendation once (fast draft is fine).
Ask the assistant to re-run the same question with the Evidence-Gate structure and your current Context Pack.
Copy section C) Unverified gaps into your next-actions list.
Choose one gap and update the system (project note, meeting note, task, or reference).
Your goal is not “perfect answers.” Your goal is answers that are checkable.
What to do next
For the next 7 days, make this your default decision rule:
no decision is “final” until it has both:
a recorded final decision
at least one closed unverified gap
if you cannot close a key gap this week, explicitly mark the decision as defer with a question owner and date
This turns Evidence-Gate from a one-time check into a repeatable operating habit.