Software Test Coverage Confidence Standard

What kinds of test cases a software suite must contain before a green build means much. Declare a target mix for the product, classify every test once, and compare actual coverage with the target—by case type and by feature.

Suite composition

Every test case receives exactly one primary case type. Primary types are mutually exclusive, so a suite’s composition always totals 100%. Other properties a case exercises are kept as secondary tags and never double-counted. When several types apply, the first in this precedence wins: Security → A11y → Performance → Negative → Boundary → Errors → State → Positive.

Positive / expected behavior

The happy path: valid input produces the expected result.

State transitions

Lifecycle, workflow and status changes, including invalid transitions.

Error handling & recovery

Failures, exceptions, outages and graceful recovery.

Boundary / edge cases

Limits, empty/maximum values, off-by-one and threshold behavior.

Negative / rejected behavior

Invalid input or disallowed actions that must be rejected correctly.

Performance / load

Latency, throughput, load, stress and resource limits.

Accessibility

Keyboard, focus, labels, contrast and assistive-technology behavior.

Security

Authentication, authorization, injection, secrets and data-exposure checks.

Missing evidence is unknown, not passing. Blocked and unknown results never count as passing, and code coverage alone does not establish that a risk was tested.

Target profiles

Targets depend on the application. These proposed starting profiles are configurable; set the mix before evaluating, then compare actual composition against it. A zero is no floor, not a ban.

Profile	Positive	State	Errors	Boundary	Negative	Performance	A11y	Security
Web / UI application User flows plus invalid input, limits, failures, state changes, security and accessibility.	Positive 30%	State 10%	Errors 10%	Boundary 15%	Negative 20%	Performance 3%	A11y 5%	Security 7%
API / backend service Contracts, validation, error codes, auth and resilience.	Positive 30%	State 5%	Errors 15%	Boundary 15%	Negative 25%	Performance 0%	A11y 0%	Security 10%
Library / SDK Valid and invalid inputs, edge values and error contracts.	Positive 35%	State 0%	Errors 15%	Boundary 25%	Negative 25%	Performance 0%	A11y 0%	Security 0%

Gaps and compliance

For a suite of N items, the target count for a type is round(target% × N). A type is on target when its share is within the declared tolerance (default ±5 percentage points) and, for any non-zero target, at least one case exists. Types with a zero target are neither required nor limited. Two remediation plans are reported:

Add-only: the smallest number of new cases that brings every type up to its target share once the denominator grows.
Rebalance at current size: add to under-filled types and revise or retire over-filled ones—only after checking topic coverage.

Composition match = 100 − ½ Σ |actual% − target%|. It summarizes distance from the profile; it is not a quality score, a pass rate or a confidence level.

Type × feature

A full bucket can still miss a critical feature. Cross case type with feature and mark applicable empty cells as gaps; explain exclusions. Do not interpret suite composition as deployment prevalence.

Human review

Coverage says what kinds of cases exist; it does not say they are correct. Review the test cases and their execution evidence separately under the Review Confidence Standard.