# Agent Task journeys

import { Steps } from '@astrojs/starlight/components';

An Agent Task journey gives a coding agent, Claude Code or Codex, a job to do with your product, such as following your quickstart, and checks whether it got there. Use it to find out whether coding agents can build with your product from your public docs.

## How an Agent Task journey works

| Part | What happens |
| --- | --- |
| **Agents** | **Claude Code** (selected by default) and **Codex**. Chat assistants such as ChatGPT cannot take an Agent Task journey. |
| **Where the agent works** | Each run starts in a new, empty working folder. The agent runs commands and writes files there without stopping to ask for permission. |
| **What it can reach** | The public web, through its own search and fetch tools. It does not get your repositories, your private docs, or any account or API key for your product. |
| **The conversation** | The agent works through your turns in order, in one session. Each run is a separate session. |

## Before you begin

- **Journeys** in the app sidebar. See the [Journeys overview](/journeys/).
- Any member of your organization can create and run journeys.

## Steps

<Steps>
1. Open **Journeys** and click **New journey**.

2. In **What do you want to find out?**, describe the job, for example "Can someone set up a Node.js project with the Acme SDK by following our quickstart?"

3. Under **Kind**, click **Agent Task**. Under **How we'll run it**, the agents change to **Claude Code** and **Codex**.

4. Under **The scenario**, write the task in the first turn field, the way a developer would ask their coding agent. Choose a job that needs no account or API key, because the agent has none. Click **add a turn** for follow-up instructions.

5. Under **Who's attempting it**, select a persona or **Write your own**.

6. Under **What we'll check**, click **N more we can check** and add the checks listed under **Suggested for agent task journeys**, such as **it completed** and **the documented order worked**. To grade a specific outcome, click **add a custom check** and write a rubric. See [Write checks that mean something](/journeys/write-checks/).

7. Under **How we'll run it**, click **Claude Code**, **Codex** or both, select each one's model, and set **Number of runs**. There is no web search control for this kind; by default the agents can search the web.

8. Click **Try it once** to test your setup with one run per agent, then **Run N times** for the full report.
</Steps>

## How Agent Task runs are graded

These checks are suggested for Agent Task journeys; you can add any other check too.

| Check | How it is graded |
| --- | --- |
| **it completed** | Judged. Passes only if the command that produces the result ran and its output, or the file it left, confirms it. Fails if the agent only described what it would do or stopped partway. |
| **the documented order worked** | Judged. Fails if following your page's steps in their documented order caused an error that another order would have avoided. |
| **it hit no contradictions** | Judged. Fails if two pages the agent fetched disagree on something it relied on. |
| **it never gave up or asked for a human** | Measured from the final answer. |
| **it finished within the turn ceiling** | Measured against your **Turn ceiling**. |
| **it stayed under the cost ceiling** | Measured against your **Cost ceiling**. |

For judged checks, a grading model reads the commands the agent ran, the pages it fetched and its final answer. It treats the agent's own summary as a claim to check, not as evidence, and does not count a failure of the machine the agent ran on, such as a missing system package, against your docs. It quotes what made a check fail.

## Check it worked

- The journey's **Reports** tab shows a new report, marked **Running** until every run has finished.
- Click a square in the report's scorecard to open a run. The run page shows the agent's activity step by step, its answer, and each check's result with the reason. A run that errored shows its error message instead.

## Limits

- **Public pages only.** No repositories, private docs, accounts or API keys, so a task that needs a sign-in or a key cannot finish.
- **Codex does not report the sources it read.** Its runs record no cited pages, so the report shows its sources as not reported and a **cites a page** check always fails.
- **Turn counts.** Only Claude Code reports its own turns. For Codex, **it finished within the turn ceiling** counts the turns in your scenario.
- **Coding agents run one at a time**, so a report with many runs takes longer than one on chat assistants.

## Next

- [Write checks that mean something](/journeys/write-checks/)
- [Turn a gap into a fix](/journeys/turn-a-gap-into-a-fix/): send what went wrong to the Docs Agent.
- [Track results over time](/journeys/track-results/)