When to run this checklist
Run it once the AI step has been designed and before it is switched on: a classifier that reads replies or form messages and routes them, a model that composes or selects a line as each message is assembled, or an agent that drafts and, with permission, acts. In each, no person reads each output before it takes effect. Run it again before any change to the prompt or a shared context block, the model, the vendor or the tool's settings, the documents the model draws on, or an agent's tasks and permissions, because a step that passed with one combination says little about the next; the automation guide treats each new task an agent takes on as a new deployment.
The person who ticks it is the workflow's owner, the person named in the inventory who can pause it, with the owner of the AI use in the register, whoever approves offers and claims for the locked blocks, and whoever owns data protection for the record fields and the suppression check.
Neighbouring gates sit around this one. Every AI use first passes the intake gate of the AI governance operations playbook: tested, data minimised, tier set and register row complete. The automation governance guide holds the activation rule for any workflow: no inventory row with an owner and a measure, no activation. An assistant that answers customers or drafts agents' replies is gated by the AI customer service launch checklist; the flow around the AI step, its trigger, branches, frequency and consent, goes through the email automation flow launch checklist; and copy drafted with AI and published goes through the AI-assisted content review checklist. None of them gates the step where a model composes, classifies or acts unread, which is where this list sits.
Before you start
- The design of the step: what the model reads, what it writes or decides, and what happens next.
- The workflow's inventory row, or the blank row it will need, and the AI use's register entry and tier.
- The template, marked up to show which blocks are fixed and which lines the model may write.
- The approved variants or the brief listing the facts the model may use, and the documents it draws on.
- The record fields the step will pass to the model, with the reason each one is needed.
- The stored test set, its rubric, the baseline and the pass bar.
- The seed contacts for every branch, and a place to run the step where nothing reaches customers.
- The incident routine the team has agreed, and the change log for the go-live.
The checklist
The groups follow the order of the decisions: what the step may do at all, what the model may write, what bounds the volume, what proves the step works, and who watches it afterwards. A line in the first four groups that cannot be ticked holds the go-live; the last group can be finished on the day, but not skipped.
Scope and permissions
What the model may write
Limits
Testing
Watching and incidents
Where each check is explained
The widget keeps the lines short so they can be ticked; the reasoning lives in the guides. Each check links to the page that explains it.
| Check | Explained in |
|---|---|
| Sends, spending or record changes asked about first | Rule-based automation vs AI agents |
| Sends, spending and record changes kept as rules with approval | AI automation that sent the wrong messages |
| Agent permissions granted per action, each action logged | Agents in the AI automation guide |
| Inventory row and an owner who can pause it | Automation governance |
| Review tier set by the output's risk | Accuracy and review workflows |
| Offers, prices, claims, legal lines and footers locked | Send-time copy in the AI automation guide |
| Model confined to approved variants or a brief | AI automation that sent the wrong messages |
| One current source per fact, each with an owner | Preparing documents for an AI assistant |
| Only the record fields the task needs | Minimising data before prompting |
| Claim, banned-term and link checks before send | Guardrails in the AI automation guide |
| Rate limit that pauses for review | Guardrails in the AI automation guide |
| Dry run after launch or any change | AI automation that sent the wrong messages |
| Volume alert to the owner | AI automation that sent the wrong messages |
| Frequency cap and global suppression check before every send | Safety rails in automation governance |
| Low-confidence classifications routed to a person | Classification in the AI automation guide |
| Stored test set from real work, rubric written first | Testing prompts and models |
| Baseline and pass bar with failures never accepted | Pass bars in the testing guide |
| Seed contacts through every branch | Seed contacts in automation governance |
| Versions recorded behind each output | AI output quality getting worse |
| Test set rerun before any change goes live | Re-testing in the testing guide |
| Sampling plan with reader, cadence, log and pause threshold | Sampling plans in accuracy and review workflows |
| Reader of the agent's logs named | Guardrails in the AI automation guide |
| Outcome compared with a holdout | Measuring automation with a holdout |
| Contain, correct, tell and trace agreed | Handling an AI mistake |
If an item fails
The scope group is the first hard stop. A step that can send, spend or change records on a model's decision alone does not go live as designed. The comparison of rule-based automation and AI agents gives the questions to ask again, and the usual fix is to narrow the model rather than remove it: the model keeps reading, labelling and drafting, while sending, stage changes, offers and deletions pass to a rule or a person. A workflow with no inventory row, or no owner who can pause it, waits too.
The locked blocks are the second hard stop. An offer, price, claim, legal line or footer the model can rewrite is one it will eventually rewrite, so a template that leaves any of them open goes back to be rebuilt. A library holding two versions of the same promotion is fixed first as well, as the tutorial on preparing documents for an AI assistant shows: GroundingMaking a language model answer from specified material, such as documents retrieved for the question, rather than from what it absorbed in training. narrows what the model invents without stopping it, and it cannot correct a library that is itself wrong.
The suppression check, the frequency cap and the record fields are not lines to note for later. Whatever the model wrote, a message that reaches someone who opted out is first of all a consent problem, as the diagnosis and the tutorial on fixing a mistake in a sent email both treat it, and the guides that cite the ICO's direct marketing guidance describe it as setting out the UK rules for electronic marketing, an opt-out route in every message among them. Every send passes the suppression list check and the frequency cap, and an AI step that bypasses either does not go live. The tutorial on minimising data before prompting reads the GDPR as listing data minimisation among its principles, so a step that passes whole records goes back until it passes only the fields the task needs. This is the pages' general direction and not legal advice; the rules for each purpose are for whoever owns data protection to set.
The testing group decides the date, not whether the step happens. A missing test set is built from real work, as the guide to testing prompts and models describes, and a test set stored beside the prompt in a team prompt library makes each later rerun routine. A failure the pass bar never accepts, such as an invented price or offer, stops the step for the inputs that produced it: narrow the scope to the cases that passed, send the rest to a person or a rule, fix and rerun. A seed contact that receives the wrong thing is a gap found before a customer found it. Where the platform has no volume alert, the governance guide's fallback, a daily report that sets the day's sends against the recent norm, meets the line.
The watching group can be finished on the day of go-live, and the holdout line can go ahead with a note where the step's effect is not in doubt. But a step with no sampling plan, no named reader of its logs and no agreed incident steps will drift without anyone noticing. The accuracy and review workflows guide makes the sampling plan the review for automated tiers, and the guide to handling an AI mistake sets out the moves, contain, correct, tell, trace and log, to know before they are needed. The habit under every line comes from the automation guide's reading of the NIST AI Risk Management Framework: a model is a part with failure modes you can name, so its uses are mapped, its behaviour measured and the findings acted on.
An appliance repair service adds an AI step to its follow-up flow (illustrative)
An approval step, a locked line and a greeting caught in the dry run
Pruxanel Appliance Repairs, an invented appliance repair service, emails each customer after an engineer's visit, about 1,400 follow-ups a month. Its marketing lead plans an AI step: the model picks one of 8 approved opening lines to suit the job and classifies replies into 5 labels, so complaints reach the service desk and rebooking requests reach the diary. The first design also lets an agent rebook visits on its own. The service manager, who owns the workflow, runs the list with the marketing lead and the office manager who handles data protection.
Scope catches the agent: rebooking commits an engineer's time, so the agent now drafts the rebooking with a proposed slot and a person on the service desk confirms it, and low-confidence labels go to the same queue. The template review catches a line the brief had left open under the opening; in the first test batch the model wrote a 10% discount on the next service into 3 of 40 messages, an offer the business has never made, so the line becomes a fixed block with no offer in it. A rate limit pauses the step for review after 30 sends in an hour, more than the busiest hour of the last quarter, and the service manager is alerted when a day's sends depart from the recent average.
The test set holds 90 past replies, each with the label it should get, and the pass bar fails any complaint labelled as something else. The first run labels 84 correctly; 2 of the 6 misses are complaints read as rebooking requests, so the complaint label gets clearer examples and the rerun passes. The dry run, 150 follow-ups generated and read but not sent, shows the last catch: for landlord bookings the contact name field had merged the landlord's and the tenant's names, and 11 greetings addressed a tenant by the landlord's name. The step now receives only the first name of the person at the visit, with a fallback. It goes live with the service manager reading 25 sent messages and 25 labelled replies every Friday, logging failures by type and pausing the step if a complaint has been mislabelled, and with the prompt, model and test set versions in the change log.
Common mistakes
- Handing an agent the whole job. Qualifying, writing, sending and updating are separate permissions; grant them one action at a time, starting with drafting.
- Leaving one line of the template open. The offer nobody approved usually arrives through the line nobody thought to lock.
- Passing the whole record. Every extra field is one the model can put in a greeting, and data the task never needed.
- No route for the unknown. A classifier forced to pick a label picks a wrong one with confidence; low-confidence results and timeouts need a defined path, usually the one the workflow followed before the model.
- Testing once. A vendor's model update quietly replaces the step that passed, and the diagnosis for AI output quality getting worse shows why recorded versions are how the change is noticed.
- A launch-week sample. The sample never ends: it keeps a named reader, a log of failures and a point at which the step pauses.
- Logs nobody reads. An agent's Audit TrailThe record, attached to the work item, of what was reviewed, which version, who reviewed it, when, what they decided and the evidence relied on. is only a control if someone named reads it and can pause the agent.
- Measuring only that it runs. Where the step's effect is in doubt, a Holdout GroupA randomly chosen part of an audience kept from exposure to measure what exposure caused. shows whether it changed the outcome.
The checklists library holds the lists that sit beside this one, from the launch of an AI customer service assistant to the launch of an automated email flow.
