# CARBON opinionated testing doctrine

These are strong defaults, not universal laws. Apply them anonymously: never
name, imitate, quote, or claim to speak for an individual author. The
machine-readable source of truth is `testing-doctrine.json`.

## North star

Produce decision-relevant confidence from repeatable evidence—not ceremony,
test counts, framework loyalty, or one impressive run.

CARBON begins with user intent, system boundaries, consequential failures, and
observable outcomes. It uses code, contracts, callers, production signals, and
live behavior as complementary evidence. Generated checks remain candidates;
inspection is not execution; a green suite is not proof that the oracle is
right.

## The exploration bargain

Exploration is valuable when the product is unfamiliar, the requirement is
ambiguous, or the failure space is not yet understood. Human judgment, tours,
heuristics, and session-based work can find risks that a predefined suite will
never ask about.

The cost is weak repeatability when the mission, data, actions, state, oracle,
and observations stay in the tester's head. Person-dependent sessions are hard
to audit, compare across builds, rerun economically, or use as a release gate.
"Rapid" must not become shorthand for leaving no durable evidence.

CARBON therefore treats rapid or session-based exploratory testing as a
discovery instrument, not a complete confidence system. A useful session has a
bounded charter, test data, actions, observations, evidence, stop conditions,
and follow-up. Consequential discoveries become named candidates with a
reproducible setup and oracle. A session alone never becomes trusted regression
or release proof.

## Opinionated defaults

- Prefer intent and observable outcomes over incidental click choreography.
- Prefer repeatable automation for stable, consequential behavior.
- Use human judgment where taste, ambiguity, accessibility experience, or
  unfamiliar failure interpretation changes the decision.
- Prefer user-visible, integration, contract, and system-boundary evidence;
  use unit checks when they are the smallest precise proof.
- Treat oracles as engineered, fallible components. Review assertions,
  rubrics, judges, snapshots, and expected results.
- Score tests by information gain, fault-detection power, risk coverage, and
  production relevance—not by count.
- Automate the stable remainder of exploration, but do not preserve obsolete
  or low-information cases forever.
- Test clean state and realistic dirty state. Preserve versions, data, seeds,
  environment, actions, outputs, and evidence needed to investigate failure.
- Use exact assertions for exact contracts; use sampled evaluation and
  calibrated judgment for legitimate variation.
- Keep generation and validation separable. AI can propose and execute, while
  people retain authority over destructive actions, external writes, trusted
  promotion, and release.

## Practices in balance

Every practice has a useful region and a failure mode. The `spices` array in
`testing-doctrine.json` carries the operational pro, con, and CARBON default for:

- rapid and session-based exploration;
- scripted repeatable checks;
- manual and human evaluation;
- unit and end-to-end testing;
- mocks and stubs;
- coverage metrics and fixed test-pyramid ratios;
- shift-left plus production observability;
- randomized, fuzz, and property-based testing;
- snapshots and goldens; and
- AI-generated tests and judges.

The framework must surface both sides when recommending one of these practices.
It must choose from product risk and evidence economics, not from allegiance to
a testing school.

## Evidence lifecycle

`candidate → observed → repeatable → regression`

- A candidate is an idea, however good.
- Observed means a terminal result with an actual outcome and evidence.
- Repeatable means the current definition has worked under a reproducible
  protocol in distinct observations. For nondeterministic behavior, this means
  repeatable measurement of a distribution—not identical wording.
- Regression requires explicit human review and promotion.

## Anti-patterns

- Treating exploration, generated tests, or source inspection as a pass.
- Buying confidence by test count, line coverage, framework adoption, or
  certification.
- Automating the implementation's answer without challenging the oracle.
- Blind reruns that preserve neither context nor a learning objective.
- Testing selectors and mocks while the real user outcome or integration fails.
- Generalizing from one happy path, one clean machine, one judge, one score, or
  one impressive run.
- Applying rigid layer ratios, testing schools, or ceremonies without regard to
  architecture, risk, cost, or observed escapes.

## Application rule

For every plan, state the population, risk, oracle, evidence channel, stop
condition, and uncertainty. For every recommendation, explain the useful case,
the failure mode, the chosen counterweight, and what evidence would change the
decision.
