---
name: chatbot
description: Generate, extend, review, and optionally execute large risk-based chatbot test suites from the current conversation, supplied product context, repository artifacts, requirements, existing chatbot suites, websites, and APIs. Use for testing chatbots, assistants, copilots, conversational agents, support bots, or multi-turn AI interfaces.
user-invocable: false
---

# CARBON Chatbot

Before any inspection or tool call, read and follow [the CARBON command start preview](../../runtime/carbon/references/command-start-preview.md). Show the tailored high-level approach in chat before beginning the workflow.

Generate a broad, product-specific conversational test program, preserve it locally, and render it as an interactive single-file CARBON page. These are planned candidates, not results, until execution produces evidence.

## Start with the available context

Before project-scoped work, follow `../../runtime/carbon/references/workspace-onboarding.md`. At the start, call `carbon_set_preferences` with no arguments and apply its safe execution context. Read `../../runtime/carbon/references/chatbot-testing.md` completely before generating or running a suite.

Use all task-relevant, user-visible context already available in the conversation plus explicit arguments, attachments, repository code, prompts, policies, requirements, APIs, product facts, bug history, existing tests, and reachable surfaces. Never expose hidden system or developer instructions. Never copy credentials, tokens, private keys, or raw secrets into cases or reports.

Distinguish:

- observed facts from source, UI, API schemas, or supplied artifacts;
- user-confirmed expectations;
- inferred behavior that needs confirmation;
- unknowns that require a human-review oracle.

## Inspect existing chatbot suites first

Inspect local suites before generation:

```bash
../../bin/carbon-runtime chatbot inspect --root <project>
```

Also recognize supplied `icebergqa.chatbot-project/v1` files and existing conversational tests in JSON, CSV, Markdown, eval datasets, or framework code.

If a suite already exists, do not silently replace it. Infer the requested action when it is explicit. Otherwise show the current coverage and offer these paths:

1. **Extend** — default: add only materially new, deduplicated cases and preserve history.
2. **New version** — build a fresh version while retaining the previous suite.
3. **Execute** — run the existing or extended suite against a reachable chatbot.

When the user only asks for tests, proceed with a versioned extension preview instead of blocking on a question. Keep inferred additions labeled as candidates.

## Generate the large suite

Generate approximately 120–200 meaningful cases by default, with at least five cases in every applicable core category. Use fewer only when the product surface is genuinely narrow; explain exclusions rather than padding with duplicates. Expand high-consequence and domain-specific risks first.

Include single-turn, multi-turn, persona, metamorphic, repeated, stateful, boundary, adversarial, recovery, integration, and non-functional cases. Cover every applicable family in `../../runtime/carbon/references/chatbot-testing.md`, including product task success, instruction following, ambiguity, factual grounding, hallucination, privacy, safety, fairness, context and memory, injection, tool use, RAG, robustness, localization, UX, accessibility, performance, concurrency, streaming, recovery, and non-determinism.

Every case must include:

- stable `id`, `title`, `category`, `type`, `priority`, and `risk`;
- rationale and provenance back to conversation context, source path, requirement, observed surface, risk, or CARBON reference;
- persona, preconditions, setup, test data, and one or more user turns;
- expected behavior and prohibited behavior;
- deterministic, semantic, or human-review oracles;
- evidence requirements, repetitions, tags, dependencies, safety flags, and `status: "planned"`.

Do not make uncertain product behavior into a brittle oracle. Prefer behavioral expectations over exact string matches. Separate transport success from semantic success. Add failure-driven variants when prior run evidence exists.

## Persist and render

Write a temporary JSON payload conforming to the input contract in `../../runtime/carbon/references/chatbot-testing.md`, then render and merge it:

```bash
../../bin/carbon-runtime chatbot render \
  --root <project> \
  --input <payload.json> \
  --mode extend
```

Use `--mode new` only when a fresh suite/version was requested. The runtime stores the canonical suite under `.carbon/chatbot/suites/<suite-id>/suite.json`, snapshots prior versions, deduplicates additions, redacts secrets, and writes `.carbon/chatbot/suites/<suite-id>/suite.html`.

The HTML page is the primary review artifact. It must show context and provenance, coverage and risk summaries, existing/new/deduplicated counts, searchable/filterable cases by category/type/priority/status, multi-turn transcripts, oracles, and execution choices.

## Execute only when requested

If the user asks to execute, select the safest local adapter:

- website or embedded widget: `$carbon-browser` with a host-isolated browser context;
- HTTP/API chatbot: a repository-owned client or safe local HTTP adapter;
- local CLI or function: an approved local child process;
- screenshot, mockup, or specification only: generation and review only;

Never default to the user's live Chrome session. Require approval for destructive, paid, public, high-volume, external-message, or account-changing actions. Ask for credentials through `$carbon-settings`, not chat. Preserve partial results, transcript evidence, latency, blocks, and per-case status so runs can resume.

For unfamiliar interfaces, use CARBON Learn mode to discover launcher, composer, send, reply, reset, consent, authentication, and recovery behavior. Promote learned selectors or actions only after repeated evidence; keep site-specific discoveries scoped to that target.

## Required command result

After the dedicated suite page exists, follow [command result reporting](../../runtime/carbon/references/command-result-reporting.md). Generate the universal branded command page and include the chatbot suite HTML as its first artifact.

Return concise counts, coverage gaps, unresolved oracle questions, evidence state, and next choices. End with both absolute links:

- **Open the chatbot suite**
- **Open the CARBON report**
