CARBON — Engineering ConfidenceUser Guide
CHAPTER 38 / 60 Pro

/carbon-skeptical-review

Independently challenge AI-system claims, eval plans, and release evidence for hidden assumptions, unrepresentative samples, weak oracles, severe failures hidden by averages, and confidence theater.

Jay · Your AI test managerProof & release confidence

When to use it

A readiness claim feels stronger than the evidence.

Before you start

Ask which assumptions, omitted slices or weak oracles could reverse the conclusion. Require counterevidence and open questions, not a rhetorical critique or fabricated execution.

Try this in your coding agent.

/carbon-skeptical-review Challenge our claim that the search assistant is ready based on these eval results.

Replace sample paths with your project. Slash names are shorthand; your host may use a plugin prefix or a skill picker. Invocation help →

ILLUSTRATIVE WALKTHROUGH

Release claim challenge

An expanded teaching example, not a recorded or live execution. The report stays visible while the replay explains how to read it.

  1. 01Read context
  2. 02Analyze
  3. 03Show evidence
  4. 04Report

Coding-agent conversation

Read context

I’m reviewing release claim challenge. I’ll keep the scope and permissions explicit.

Analyze

Analyze: Challenge our claim that the search assistant is ready based on these eval results.

Show evidence

Here is the evidence trail to inspect: Compare the readiness claim with eval cases: refund disputes have no negative controls.

Report

The headline score hides an untested high-risk slice. Review challenges evidence; it does not run the missing cases.

CARBON

Example report

Illustration
Answer vs supporting source
Answer vs supporting source — TestBucks teaching screen

TestBucks course screen with illustrative analysis. The screenshot supplies context; the demo does not establish a newly observed bug.

  1. Question / intent
  2. Reference evidence
  3. Scoring & uncertainty

Illustrated outcome

The headline score hides an untested high-risk slice.

Evidence to inspect

Compare the readiness claim with eval cases: refund disputes have no negative controls.

Still untested / not established

Review challenges evidence; it does not run the missing cases.

Ready. Read the full example or play its steps.

What a useful result contains

A bounded claim

For release claim challenge, state the target, revision, environment and actual work performed. Keep planned, executed, blocked and untested work separate.

An inspectable trail

Compare the readiness claim with eval cases: refund disputes have no negative controls. In a real run, retain the source, trace, screenshot or fixture behind the conclusion.

The qualification

Review challenges evidence; it does not run the missing cases.

The next decision

Decide whether the evidence calls for a focused check, a reviewed fix or more context. Do not convert a plausible example into a verified finding.

Adapt the example to your project

  1. Replace release claim challenge with a real feature, path or decision in your project.
  2. State the expected rule and its source. Include safe fixtures, constraints and a budget where appropriate.
  3. Ask for exact evidence and remaining gaps. Approve consequential actions separately.
/jay Review the /carbon-skeptical-review result. Separate observed evidence, assumptions and untested scope, then recommend the next useful check.

Where to go next

/carbon-eval-design-review →/carbon-confidence →

Content checked against the local command definition: skills/carbon-skeptical-review/SKILL.md. Examples adapted from the homepage’s TestBucks walkthroughs. Availability is edition- and host-dependent.