Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Checklists

AI Customer Service Launch Checklist

A go-live checklist for an AI assistant or reply drafting tool in customer service: scope, approved content, handover, disclosure, data, testing and review.
Checklistintermediate10 min readUpdated

When to run this checklist

Run it before an AI assistant starts answering customers in chat, messaging or email, and before an AI tool starts drafting replies that agents edit and send. Run it again before the assistant's scope changes (a new contact reason moves from assist to automate), before its content source changes (a help centre is restructured or a new policy library is connected), and before its model or prompt changes, because a test passed on one combination of model, prompt and content says little about the next. Passing this list is also the entry condition for the AI chat assistant test, which compares assistant-first and staff-only product-page chat once the assistant is ready to answer. The academy exercise on scoping, gating and testing a coffee machine retailer's chat assistant practises sorting the retailer's contact reasons by risk, splitting owned saved replies from drafts and writing the test this list opens, judged on contribution per chat visitor rather than chats closed without staff.

The customer service lead runs the list, with the owner of the help content for the content lines and whoever owns the business's AI use policy for the scope and data lines. A vendor or implementation partner can do much of the work, but the ticks belong to the people who will answer for the assistant's replies.

Two neighbouring lists sit on either side. The governance checklist in the disclosure and data privacy guide settles disclosure, the data that may enter a tool, vendor terms and retention for any AI use; run it first, and this list points to it rather than repeating its data and vendor lines. The AI-assisted content review checklist gates copy that is published, not conversations. Where the assistant sits inside an automated journey, such as an order confirmation that invites a chat, the email automation flow launch checklist covers the flow itself. How to build a test set and rubric is taught in the guide to testing prompts and models; this list only asks whether that test was run.

Before you start

  • The list of contact reasons support already tags, with a note of which ones are complaints, refunds, exceptions, legal matters or likely to involve a customer in a vulnerable situation.
  • The help content the assistant will answer from, with each article's owner and last review.
  • The written service tone, including approved and rejected example replies, built on the brand voice.
  • The business's AI use policy and its register of approved uses and owners.
  • The handover design: how a customer reaches a person, what the person receives, and what happens out of hours.
  • The integrations the assistant will use to read account, order or booking data.
  • The test set, rubric, pass bar and results from testing the assistant on real past questions.
  • The quality scorecard the team uses to review human conversations.

The checklist

The groups follow the order in which the decisions are made: what the assistant handles, what it answers from, how customers leave it and know what it is, what data it sees, and how its answers are tested and watched. Scope comes first because every later line depends on which contact reasons are in play.

0/21

Scope

Content and answers

Handover and disclosure

Data

Testing and review

Where each check is explained

The widget keeps the lines short so they can be ticked; the reasoning lives in the guides. Each check links to the page that explains it. The tutorial on writing the standing instructions for an AI assistant turns the scope, content, handover, disclosure and tone lines into the written instructions the assistant follows, with a test conversation for each.

CheckExplained in
Each contact reason sorted into automate, assist or keep humanScoping service work in the AI customer service guide
Complaints, vulnerable customers and legal matters routed to peopleKeep-human tasks in the AI customer service guide
Scope started from the contact reasons support already tagsTagging support contacts by reason
Use and owner recorded under the AI use policyWriting a marketing AI use policy
Answers only from approved help content, citing the articleAnswering from approved content
An owner for every help article, updated before policy changesSupport channels and self-service
Source library current, with no conflicting versionsHow grounding and retrieval work
Service tone written with approved and rejected examplesWriting customer service tone guidelines
Agents write any reply that commits a refund, date or exceptionDrafting replies for agents
A person reachable at any pointChatbots and escalation in the support channels guide
Full conversation passed at handoverHandover to a person
Automated assistant disclosed at the start and when askedDisclosure and data privacy for AI in marketing
Honest message when no person is availableHandover to a person
Account access only through read-only, defined integrationsAnswering from approved content
Personal data limited to what the task needsMinimising and redacting data before prompting
Personal data masked in review logsRetention in the disclosure and data privacy guide
Tested on real past questions with a rubric and pass barTesting prompts and models before you rely on them
Re-test planned for model or prompt changesRe-testing when the model changes
Weekly sample review bookedDisclosure and sampling review
Same scorecard for automated and human conversationsReviewing support conversations for quality
Service quality and repeat contact reported, not deflectionMeasuring service quality, not deflection

If an item fails

The scope and handover groups are the hard stops. If any contact reason has not been placed, the assistant answers it by default, so it does not go live until every reason has a decision. If complaints, legal matters or signs of a customer in a vulnerable situation do not route to people, stop: an assistant that tries to handle a complaint usually makes it worse, and the recovery sequence in the service recovery and complaints guide depends on a person who can take responsibility. A customer who cannot reach a person, or who has to repeat everything after reaching one, is the failure customers remember, so a broken handover also stops the launch.

Disclosure is not a line to note for later. Both the AI customer service guide and the disclosure guide read the European Commission's overview of its approach to AI as expecting people to be told when they are interacting with an AI system, and the FTC's business guidance as holding that deception standards do not change with the technology. An assistant that does not say it is automated does not go live. That is general direction from those pages, not legal advice.

In the content group, an article without an owner or a library with two versions of the same policy is the line most often found failing, and the one that turns into wrong answers delivered confidently. Grounding narrows invented answers to what the library says; it cannot fix a library that says the wrong thing. Fix the library, then re-run the affected test questions. A tone guide still in draft can go ahead with a note if agents review every draft, but not for automated answers. The academy checkpoint on scoping assistants, owning replies and reading chat tests tests these lines at a ski and snowboard hire shop, starting with a supplier who wants its largest queue automated while two versions of one policy article are still live.

The data group stops the launch when the assistant can change account data, or can see more personal data than the task needs. Account and order data is first-party data collected for service; the minimising data tutorial shows how to strip it back.

A failed test means the assistant does not go live for the failing contact reasons. Narrow the scope to the reasons that passed, move the rest to assist, and re-test after the fix. The review lines can be finished on the day, but the first weekly sample must be booked before switch-on, with a named reviewer. The NIST AI Risk Management Framework, as the AI use policy tutorial describes it, pairs measuring how a system behaves in practice with managing what is found; a review nobody has booked does neither. If the report still leads with deflection, rewrite it before launch: Goodhart's law explains why a target the assistant can hit by being hard to get past stops measuring service, and a repeat-contact guardrail metric keeps the report honest.

A furniture retailer switches on an assistant (illustrative)

Illustrative example — not a real result

Delivery and assembly questions, with complaints kept human

Thexilo Furniture, an invented online furniture retailer, plans a chat assistant for its two busiest contact reasons: delivery status and assembly questions. The customer service lead runs the list with the help centre owner and the operations director, who owns the AI use policy. Scope passes after one change: damaged-delivery reports had been tagged under delivery status, so the team splits them into their own reason and keeps them human, because each one is a complaint that may need a replacement or refund. The test set of 120 real past questions fails the content group. Eleven answers about returning flat-pack items quote a returns article that was replaced eight months earlier but never unpublished, so the assistant has two versions to choose from. The help centre owner removes the old article, names an owner for each of the 34 delivery and assembly articles, and the affected questions are re-run and pass. The handover passes: customers can type for a person at any point, and agents see the full chat. The data lines pass because the order lookup is read-only. Before switch-on, the lead books the first weekly sample review of 40 automated conversations plus every handover, scored on the same scorecard as human replies, and the launch report tracks repeat contact on delivery questions within a week rather than the number of chats closed.

Common mistakes

  • Scoping by volume instead of risk. The commonest contact reason is not automatically safe to automate.
  • Treating the help centre as finished. The assistant repeats whatever the content says, including the article that should have been retired.
  • Designing the handover last. A route to a person that customers have to fight for turns the assistant into the reason for the complaint.
  • Leaving disclosure to the vendor's default greeting. Check what the customer actually sees at the start and when they ask.
  • Letting drafts commit the business. Refunds, dates and exceptions written by a draft and sent unread become promises nobody decided to make.
  • Testing once. A model or prompt change after launch quietly replaces the assistant that passed.
  • Reviewing only automated conversations. Without the same scorecard for human replies there is nothing to compare the assistant against.
  • Reporting deflection. Customers who give up look like savings until they leave.

The checklists library holds the lists that sit beside this one, from AI-assisted content review to the launch of an automated email flow.

Frequently asked questions

Does this checklist apply to a tool that only drafts replies for agents?

Yes. A drafting tool never speaks to customers directly, but its drafts carry the same content, tone and data questions, and agents under time pressure can start sending them unread. The handover and disclosure lines matter less when a person sends every reply, but the scope, content, data and review lines apply in full, and the line about agents writing any reply that commits the business matters most of all.

We already ran the AI disclosure and data checklist. Do we need this one too?

They answer different questions. The disclosure and data checklist is the governance gate for any new AI use: who is told, what data goes in, what the vendor terms say and how long things are kept. This list is the service gate for an assistant that answers customers: which contact reasons it handles, which content it answers from, how customers reach a person, and how its answers are tested and reviewed. Run the governance list first, then this one, and treat its data lines as already settled where they overlap.

What should we report once the assistant is live?

Report what customers experience, read by contact reason: whether issues were resolved without the customer coming back, whether sampled answers matched current approved content, how easily customers who wanted a person reached one, and satisfaction or effort scores. Avoid reporting deflection, the share of contacts that never reached a person, as the measure of success, because an assistant that is hard to get past scores well on it while service gets worse.

Terms used on this page

Frameworks

Where this page is used

Part of these learning paths

Also related

Sources

  1. Regulatory framework proposal on artificial intelligence — European Commission
  2. FTC business guidance — Federal Trade Commission
  3. AI Risk Management Framework — NIST

Related