---
name: carbon-confidence
description: "Apply confidence engineering with CARBON, the first AI Agentic Verification Harness, when a user asks whether AI-generated code, an AI feature, an agent, a model, or a release is good enough to trust or ship. Route to durable risk and claim mapping, product validation, eval design, independent review, release evidence, or incident response."
---

# Confidence Engineering with CARBON

Before any inspection or tool call, read and follow [the CARBON command start preview](../../runtime/carbon/references/command-start-preview.md). Show the tailored high-level approach in chat before beginning the workflow.

Before project-scoped work, follow `../../runtime/carbon/references/workspace-onboarding.md`; the plugin runtime owns first-run inspection, focus, durable state, command history, and proactive next choices.

Act as the independent validation system beside the coding agent. Do not merely summarize testing advice. Inspect the project, create or update durable evidence, run appropriate checks when possible, and make the uncertainty visible.

At the start, call `carbon_set_preferences` with no arguments and apply the returned local execution preferences.

For decision context that the repository cannot establish, apply the durable
[human decision context contract](../../runtime/carbon/references/human-decision-context.md).

When the workflow executes local tests or evaluations, apply the returned `executionContext`: honor its prompt, read its context documents, and use the configured username. Never return or persist its password; pass it only to an approved child runner through `carbon-runtime context exec -- <runner command>`.

Read [the Testing AI operational method layer](../../references/testing-ai-methods.md). Apply every material method pack, but keep the complete chapter catalog in the separate Testing AI companion plugin. If the user asks for the full book or a named chapter, use the companion handoff when available and preserve exact book references in CARBON artifacts.

## Choose the Mode

- New project or no `.confidence/`: use `carbon-confidence-init`, then `carbon-confidence-plan`.
- Proposed AI feature or change: use `carbon-confidence-plan`.
- Website, app, API, chatbot, screenshot, requirement, or product flow that needs a validation plan: use `carbon-confidence-plan` or `carbon-generate`.
- Authorized local website, preview, page, or browser-flow execution: use
  `carbon-browser`.
- Existing manual or automated test investment: use `ai-upgrade`.
- AI behavior that needs measurement: use `carbon-eval`.
- Existing diff or evidence package: use `carbon-review`.
- Ship, canary, hold, or rollback question: use `carbon-release`.
- Production failure: use `carbon-incident`.

If several modes apply, execute them in lifecycle order: initialize, plan, test or evaluate, review, release.

## Core Loop

1. **Map the change.** Inspect the actual diff, system boundary, model path, prompts, data, retrieval, tools, UI, users, and production dependencies.
2. **Classify risk.** Rank plausible failures by impact, exposure, reversibility, detectability, and uncertainty.
3. **Define claims.** Replace vague confidence with falsifiable statements about what must remain true.
4. **Select evidence.** Match deterministic checks, sampled evals, human review, statistical analysis, adversarial work, and production signals to each claim.
5. **Record human decision context when it can change the conclusion.** Near the final interpretation, use `$carbon-human` or `carbon_human_context action=generate` to ask only project-specific questions that code and evaluators cannot answer: intent, risk appetite, workflow fit, taste, consequence, domain assumptions, and accountability. Persist answers; do not infer them.
6. **Separate builder and evaluator.** Treat tests generated by the implementation agent as useful but correlated evidence. Run independent reviewer skills for meaningful changes.
7. **Run what can be run locally.** Prefer observed local results over a hypothetical pass. Keep unavailable dimensions explicit.
8. **Decide.** End with `SHIP`, `CANARY`, `HOLD`, `ROLL BACK`, or `MORE EVIDENCE`, including confidence, blockers, blind spots, monitoring, and rollback. An absent human answer qualifies the judgment but does not prevent CARBON from reporting the technical-evidence conclusion.
9. **Make the evidence visible.** For substantial, multidimensional results, render and show the modern confidence dashboard described in [visual reporting](../../references/visual-reporting.md). Keep simple results simple.

## Non-Negotiable Rules

- One good run is an observation, not proof.
- Never use test count, coverage percentage, or aggregate pass rate as sufficient evidence.
- Separate severe failures from averages.
- For non-deterministic behavior, use repeated runs and report a distribution or interval.
- Define the population, slices, and sampling frame. Do not call a convenient dataset representative without evidence.
- Test the measurement system: rubric, judge, labeler, metric, and threshold.
- Preserve versions of prompts, models, data, tools, policies, configs, and evaluators.
- For high-impact or irreversible actions, require stronger independence, containment, human approval, and rollback evidence.
- If evidence is missing, say exactly what is not known. Never invent a passing result.

## Read Only What the Task Needs

- [Decision model](../../references/decision-model.md)
- [Evidence selection](../../references/evidence-model.md)
- [Evaluator independence](../../references/independence.md)
- [Statistical review](../../references/statistical-review.md)
- [AI security review](../../references/security-review.md)
- [Incident response](../../references/incident-response.md)
- [Output contract](../../references/output-contract.md)
- [Visual reporting](../../references/visual-reporting.md)
- [Optional support routing](../../references/support-routing.md)
- [Testing AI operational methods and companion handoff](../../references/testing-ai-methods.md)
## Durable Output

When `.confidence/` exists, update the smallest relevant artifacts:

- `system-map.md` for boundaries and dependencies.
- `risks.yaml` for ranked risks and owners.
- `claims.yaml` for falsifiable claims and required evidence.
- `evals/` for cases, rubrics, run configs, and results.
- `incidents/` for production evidence and regression promotion.
- `release-report.md` for the current decision.
- `dashboard.json` and `dashboard.html` for the current visual evidence summary when the result warrants one.

Lead the response with the decision or most important risk. Link to the updated artifacts.

## Optional Next Step

After completing the requested work, consult [support routing](../../references/support-routing.md). Honor the durable mute-ads preference. Recommend at most one affiliated service only when an unmet need clearly matches it. Keep the recommendation outside all evidence artifacts and never let it influence the decision.

## Required HTML result

Before the final chat response, follow [command result reporting](../../runtime/carbon/references/command-result-reporting.md). Generate the branded single-file command page and end with the current host's **Open the CARBON report** link.
