Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

AI in Marketing › Intermediate

Evaluating an AI Tool for a Holiday Park Operator

Practise choosing an AI tool on evidence rather than on its demo. You weight the criteria, score two invented candidates for a holiday park operator's guest emails on fit, data handling, integration and total cost, choose one and design a pilot with a success measure, an end date and a rule that decides what happens next.
ExerciseIntermediate levelModule 3: Evaluating and rolling out AI tools5 minUpdated

Why this matters

AI tools are sold on demos, and fluent copy says nothing about whether a tool reads the right guest data or respects consent. The guide to choosing and evaluating tools puts integration, identity and data handling at the heaviest weights because they cannot be fixed after purchase. The data flow and integrations guide explains why: a tool that keys on the wrong identifier or arrives by file upload becomes a manual process that eventually stops.

The operator and its shortlist

Pinecrest Holiday Parks is an invented operator of lodge and caravan parks. Its booking system is the system of record for guests and consent; there is no customer data platform (CDP). The email platform already receives bookings through a native connector. A marketing officer writes pre-arrival, post-stay and re-booking emails by hand.

Illustrative example — not a real result

Pinecrest: evidence gathered on the two candidates

CriterionWeightCandidate A: AI drafting feature inside the current email platformCandidate B: standalone AI personalisation service
Use-case fit3Drafts and varies copy within templates; no per-guest product choiceWrites each email per guest, picks park activities by party
Integration3Uses the existing booking connectorNightly file export from the booking system, to be built
Identity and consent3Reads the platform's contact record and consent flagsKeys on email only; the export carries no consent field
Data handling3Inputs not used for training under current terms; processing region coveredInputs used to improve the service unless a higher tier is bought
Usability for the owner2Marketing officer used it unaided in the trialNeeded vendor help to change a template
Reporting2Results in the platform's existing reportsOwn dashboard; no export of results to analytics
Governance2Approval step before send; change historyApproval flows and versioned prompts
Vendor1Established; feature labelled as newYoung company; responsive support
Total cost3See cost tableSee cost table
Exit1Templates stay in the platformConfiguration not exportable
Two-year cost itemCandidate ACandidate B
Licence£75 × 24 = £1,800£400 × 24 = £9,600
Set-up and integration£40040 hours × £60 = £2,400
Maintaining the nightly export£02 hours × 24 × £60 = £2,880
Training£300£300
Total£2,500£15,180

Pinecrest's marketing officer spends about 2 days a month on these emails, at about £30 an hour.

Scoring the tools

Weight the criteria before scoring, from what cannot be fixed later. Score each candidate from one to five with the evidence beside each score, and say where evidence came from a demo rather than a trial. Then apply the guide's rule: a candidate that fails an essential requirement does not win on the sum of the rest. For Pinecrest, consent read from the system of record and guest data kept out of a vendor's training are essential.

Task

  1. Mark which requirements are essential, important or useful, and justify the essentials.
  2. Score both candidates against the weighted criteria and total them.
  3. Compare two-year cost, adding any workarounds the integration checks reveal.
  4. Choose a candidate and write the decision record's key lines, including known gaps.
  5. Design a time-boxed pilot: use case, data, success measure, holdout, end date and decision rule.
  6. Say which onboarding steps must happen before real guest data reaches the tool.

Worked answer

Essentials. Consent read from the booking system; guest data not used for vendor training; results returned to the platform's reports so outcomes can be measured. Candidate B fails the first two outright.

Illustrative example — not a real result

Pinecrest: weighted scores out of 115

CriterionWeightA scoreA weightedB scoreB weighted
Use-case fit339515
Integration351526
Identity and consent351526
Data handling341226
Usability24836
Reporting24824
Governance23648
Vendor14433
Total cost351526
Exit13311
Total9561

Time currently spent: 2 days × 7.5 hours × £30 = £450 a month. If A halves it, the saving is £225 a month, or £5,400 over two years against £2,500 of cost: a return on investment (ROI) of (£5,400 − £2,500) ÷ £2,500 ≈ 116%, before any change in bookings.

Choice and record. Candidate A, on the essentials and on cost. B's better fit is real but rests on the demo and on a nightly export that would key on email alone and lose consent, exactly the identifier mismatch and consent leakage the data flow guide warns against. The record notes the gap: no per-guest activity choice. B could be revisited with a higher tier and a governed feed.

Pilot. Post-stay re-booking emails at two parks for eight weeks, with a fixed end date. Departing guests are split at random: half receive AI-drafted, human-approved emails, half the current hand-written template. The success measure is time per email and defects found in review, with re-booking rate as a guardrail KPI that must not fall below the control. The case rests on time saved, because a pilot this small cannot reliably detect a booking lift. Decision rule, written before the start: adopt if time per email falls by at least a third with no rise in defects and no fall in re-bookings; extend once if results are mixed; stop otherwise.

Before real data. The marketing tool onboarding SOP applies even to a pilot that touches real guest data: approval with a named owner, a data and privacy review with the data protection lead, access by named account, a test of consent handling and training before editing rights. Send only the fields each email needs. The privacy review gives general direction, not legal advice.

Common mistakes

  • Scoring from the demo. Fit looked strongest for B and was the least proven.
  • Comparing licence prices. B's export and its upkeep cost more than its licence difference suggests.
  • A pilot with no end date or rule. It runs until someone likes it.
  • Measuring what the pilot cannot detect. A small booking lift needs far more guests than eight weeks provide.

Next, take on the AI adoption plan capstone or return to the intermediate AI in marketing level.

Pages this lesson applies

Terms used on this page

Also related

Related