Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

AI in Marketing › Advanced

Capstone: An AI Operating Model for an Online Course Provider

The closing project of the advanced AI in marketing level. You write a year's AI operating model for an invented online course provider that adopted AI in pieces: where its course facts and voice live and who owns them, the incident procedure, the claims and endorsement rules, the AI rules in its two agency agreements, a skills plan from a baseline, a measurement design for each team and the review rhythm that keeps the model alive. The deliverable is one document with a named owner for every part.
Capstone projectAdvanced levelModule 3: Proving the value and leading the programme9 minUpdated

Brief

Krovelby Learning is an invented provider of online professional courses in accounting, project management and data skills. Its marketing team is a head of marketing, two content marketers, a paid media manager, a community and support lead and a junior marketer. The intermediate adoption plan was written and followed, so chosen use cases, ad guardrails, review tiers, a data rule and one adopted drafting tool are in place.

Since then AI has grown in pieces rather than by plan. A course-support assistant on the website answers learners' questions from a document library that nobody owns and that a vendor connected at set-up. The content team drafts course pages, and the outcome claims on them, with the drafting tool. The paid media agency has switched on the AI enhancements in both ad accounts. The creative agency used a generated presenter in an advert, and learners wrote in to ask whether she taught the course. The managing director has read last quarter's incident notes and wants one operating model that says who owns each part, how mistakes are handled, what the agencies may do, how the team's skills grow and how anyone will know whether AI is helping. She has asked for owners and dates, not principles.

Inputs and constraints

Illustrative example — not a real result

Krovelby Learning: the assistant's library and its test run

Source connected to the assistantDocumentsWhat the list shows
Course pages94Prices and start dates in a sidebar block; 11 pages for retired courses still live
FAQ articles313 versions of the refund policy
Tutor biographies406 tutors who have left
Policy pages6Accreditation terms with the exceptions in a footnote
Shared folder "support bot"41Old price lists, draft syllabuses, a learner export with names and employers

Test set of 50 real learner questions: 34 answered correctly with a source the lead could open; 9 wrong, of which 6 gave a price or start date from an old sheet and 3 described a superseded refund policy; 7 answered "I don't know" where a current document existed. The standing instructions were written by the vendor and have no hand-over list.

Illustrative example — not a real result

Krovelby Learning: last quarter's incident notes

IncidentFound byWhat happened nextStatus
The assistant quoted an early-bird price that had ended 9 days earlier to 23 learnersA complaint on day 10Price honoured for the 23; nothing changed in the libraryClosed
A course page claimed "92% of graduates promoted within a year"The paid media manager, before an ad used itNo evidence found; line cut from the page; still in 2 nurture emailsOpen
The creative agency's advert used a generated presenter; 4 learners asked whether she taught the courseThe support inboxAgency called it standard practice; advert still runningOpen
The paid media agency's automatically created assets named a certification the course does not conferThe monthly paid media review1 asset paused; the setting is still on across 3 campaignsOpen
Illustrative example — not a real result

Krovelby Learning: agencies, self-reported hours and two quarters of output

The paid media agency is on a monthly retainer with admin access to both ad accounts and no AI clause. The creative agency works project by project on a scope of work that says nothing about generated elements. Both agreements renew at the start of next quarter. Each content marketer reports saving 10 hours a week; the support lead reports that the assistant "handles 60% of questions". No baseline was taken for any task before the tools arrived.

MeasureTwo quarters agoLast quarter
Course pages drafted / accepted140 / 128260 / 190
Pages sent back at least once12%34%
Corrections after publication29
Head of marketing's review hours a week414
Assistant conversations4,1006,800
Conversations handed to a person38%29%
Complaints about a wrong answer314
Paid media spend / enrolments the platforms attribute£48,000 / 410£51,000 / 430
Enrolments recorded in the course platform395402

Constraints you must work within:

  • Any claim about course outcomes, pass rates or careers needs evidence on file before it is published, in the approved wording and nowhere else.
  • Learners are told when they are talking to an assistant, and no generated person may be presented as a tutor, learner or expert.
  • Learner records are personal data and go only into tools whose terms the data protection lead has approved; treat the model's data rules as general direction to confirm with that lead, not legal advice.
  • No new tools this year; the model governs what is already in use.
  • The course platform is the system of record for enrolments; platform-attributed figures are not.
  • The model is one document with appendices, and every part has a named owner and a date on which it is judged.

Task

  1. Decide where facts and voice live, and who owns them. Sort facts from form with the comparison of retrieval and fine-tuning, so prices, dates and policies stay in retrieval, where Grounding ties each answer to an owned passage. Clean the library with the routine in the tutorial on preparing a document library: one current source per fact, an owner and review date on each document, and the change that keeps it current. Say what the standing instructions must add, using the tutorial on writing instructions for an AI assistant, and name an owner for each, as the bicycle retailer grounding exercise does.
  2. Write the incident procedure and run last quarter through it. Set contain, correct, tell, trace and log as steps with an owner for each, from the guide to handling an AI mistake. Run each incident in the notes through it as the opticians chain incident exercise does, down to the creation and escape causes and the one thing each changes in the system. Record them in the incidents column of the AI use register, and set the review that the tutorial on reviewing a marketing error after the fix describes.
  3. Set the claims and endorsement rules. Require a Substantiation file entry before any outcome claim runs, built as the tutorial on building a claims library builds it, and make the wording check part of every review with the AI-assisted content review checklist. Apply the guide to AI claims and consumer protection to the graduates claim, the nurture emails and the generated presenter: what is withdrawn, what is labelled and who checks what from now on.
  4. Extend the policy to both agencies. For each agency settle the points the guide to setting AI rules for agencies lists, from permitted uses and the business's data to the ad account AI settings, records and exit. Write them into the agency scope of work template before the renewals, build the settings register from the tutorial on reviewing AI enhancements in ad accounts, and add a register row for each agency use.
  5. Plan the team's skills from a baseline. Follow the guide to building AI skills in a marketing team: a baseline session on real tasks, practice on the team's own work, review drills with seeded errors, champions with protected time, a library built as the prompt library tutorial describes and the tasks the junior keeps manual. Record each person on the marketing skills matrix and name the measure the programme is judged by.
  6. Design the measurement for each team. For content, support and paid media, follow the guide to measuring AI's impact on team output and the flooring retailer measurement exercise: a Baseline from records, the accepted unit and its whole cost, defect, rework and reviewer load measures, an outcome from a KPI tree and the strongest comparison each team can run. Say what the self-reported hours and the hand-over share each hide. Add a stored test set and pass bar for the course-page prompt and the assistant's question set, from the guide to testing prompts and models, with re-test triggers.
  7. Set the review rhythm and the owners. Make the register the spine of the model, one row per use with its owner, tier and incidents. Fold the incident, agency and claims rules into the AI Use Policy with the review triggers the tutorial on writing an AI use policy sets, assign each part of the model on a RACI matrix, put decisions that change a use in the marketing decision log, and fix the quarterly review with the managing director at which the measures are read and the model is amended.
  8. Write the document. One operating model: a page per part, the owner and judgement date on each, and the appendices (library register, incident log, substantiation file, agency schedules, skills matrix, measurement design and test sets) behind it.

Evaluation criteria

CriterionWhat a strong answer showsCommon gap
Grounding and ownershipPuts facts in a retrieved, owned library with review dates and keeps voice in instructions and examples, with a named owner for eachAccepts a model trained on the brand for facts, or leaves the library without an owner
Incident procedureSets contain, correct, tell, trace and log as steps with owners, and names what each quarter's incidents changed in the systemAn apology template with no trace of causes
Claims and endorsementsRequires a substantiation file before any outcome claim is published and labels synthetic people so no learner mistakes one for a tutorTreats AI output as exempt from the rules that apply to any claim
Agency rulesSettles permitted uses, the business's data in agency tools, review, disclosure, ad account AI settings, records and exit for both agenciesA policy that stops at the team's own staff
Skills planStarts from a baseline, trains on the team's own tasks, practises review as well as prompting and protects junior judgementA vendor demo booked as training
MeasurementUses a baseline, time per accepted output, rework and outcome measures with a comparison that holds up, reported honestlyHours saved as reported by the people using the tools

A strong submission also says which of last quarter's incidents would have been caught by the model it proposes, and at which step, and what evidence at the first quarterly review would change it.

Submission checklist

0/8

When you have finished, check the model against the evaluation table and against the adoption plan from the intermediate AI in marketing level that it assumes.

Pages this lesson applies

Templates and checklists used

Terms used on this page

Related