# Test your setup

*Set up your agent · 11 of 12*

- Ten real jobs, each with a line saying what good looks like.
- A marking prompt that shows its working.
- A way to know whether a change helped, before it reaches anyone.

3 min read, about 45 to build

## What it is

A short list of real jobs you give your agent, with what good looks like for each one, that you rerun after every change. People who build agents call it an eval. You don't need the word or any software, just a note or a spreadsheet.

## Why a feeling isn't enough

You tweak your instructions, add a skill or switch models, and the next answer feels better. Maybe it is. You tried it once, on one job, while hoping it had improved, so you can't tell. Do that for a few months and you'll have changed everything and learned nothing.

There's a catch, and it's the useful part. Your test can be wrong too. Anthropic's engineers (https://www.anthropic.com/research/demystifying-evals-for-ai-agents) describe a model that "initially scored 42% on CORE-Bench", a benchmark, until a researcher read the runs and found problems with the grading and the task instructions. "After fixing bugs and using a less constrained scaffold, Opus 4.5's score jumped to 95%." Same model, different test. So when a result looks bad, read the answers before you decide the agent's the problem.

## Build your ten

1. List ten jobs you really give your agent. Start with ones that have gone wrong, and add a couple of ordinary ones so you notice if a fix breaks them.
2. Save the exact input for each, with real names and numbers scrubbed. The input has to stay the same every time, or you're comparing nothing.
3. Write what good looks like in a line or two you could mark without thinking. "Under 100 words, names the invoice number, no apology" beats "sounds professional".

| # | Job | Good looks like | Run 1 | Run 2 |
|---|---|---|---|---|
| 1 | Chase an overdue invoice | Under 100 words, names the invoice number, no apology |  |  |
| 2 | Meeting notes to actions | Every action has an owner or says [OWNER?], nothing invented |  |  |
| 3 | Reply to a complaint | Acknowledges the problem, offers one next step, promises nothing that hasn't been agreed |  |  |

## Mark them

Mark the first run by hand. Once there are too many, use a fresh chat to mark them, and make it show its working:

**The marking prompt**

```
You are marking an AI's work against a checklist. You didn't write this work, and you're not here to improve it.

Task: [THE TASK]
What good looks like: [YOUR ONE OR TWO LINES]
The answer to mark:
[PASTE THE ANSWER]

For each thing on the checklist, say pass or fail and quote the exact words that prove it.
If the checklist itself looks wrong or unfair for this task, say so.
Don't give an overall opinion of the writing.
```

Read every failure yourself. An AI marker can be wrong in the same way the first AI was, and a failure might mean your "good looks like" line needs fixing. Where a line can be checked mechanically, like a word count, use a fixed check (/guide/check-the-facts.html) instead.

## Rerun after every change

New instruction, new skill, new model: rerun the ten, and write the score and the date beside them. If the score drops, you've found out before it reached anyone. Add each new failure you meet in real work to the list, so the test keeps up with you.

## Is it working

You'll know it's working the first time you change something, rerun the ten and see a result you didn't expect. That's the point of it. Without the list, that change would have gone out on a feeling.

I learned the "your test can be wrong" part myself. I tested a store I built for my agents' facts against plain notes, published the score, and then the answers got read one at a time. The grader had been failing right answers for mentioning a trap value, so the headline number was measuring the grader. The claim came off my site on 29 October. The test was useful, and the score needed reading before anyone believed it.

[The ten-task sheet and the marking prompt](/guide/kit/checks/ten-tasks.md)
