Hypothesis
Opening the video ad with the customer's problem instead of the brand logo will lower cost per acquisition, because more of the right viewers keep watching to the offer. The first moments of a video ad decide whether someone scrolls past. A logo tells the viewer who is speaking; a problem tells them whether the ad is about them. The hypothesis bets that the second message filters in the people most likely to buy.
The test is valuable precisely because it isolates the opening. Most creative tests compare two completely different ads, which tells you that one ad beat another but not why. Here the edit after the opening is frame-for-frame identical, so a difference in results can be credited to the hook. The hooks and UGC guide and the TikTok content formats and hooks guide cover how to write strong openings; pick one problem-led opening that a customer would recognise, and write the hypothesis, evidence and exact edits into the experiment hypothesis template.
Metrics and guardrails
The primary metric is CPACost of each conversion.: spend divided by acquisitions, using the conversion event the campaign optimises for and the same attribution setting in both arms. Use acquisitions recorded by your own systems where you can, alongside the platform's figure, because the platforms model some conversions and the two counts can differ.
Post-click conversion rate is the first guardrail. If the problem-led opening draws more clicks from people who then do not convert, the landing page or the offer is not matching the hook's promise. The conversion rate calculator gives each arm's rate from clicks and conversions. FrequencyAverage times a person saw an ad. is the second: if one arm reaches a smaller audience repeatedly, its cost will drift upwards for reasons that have nothing to do with the opening. Report click-through rate and the share of viewers who watch past the opening as diagnostics; they explain the result but do not decide it.
Design
Use the platform's own split testing feature, which both Meta and TikTok document in their help centres. It divides the audience into random, non-overlapping groups and shows each group one version, so the delivery system cannot shift spend between them. Randomisation is at the level of the person or account the platform uses for this purpose, not at impression level. On TikTok, the guide to testing TikTok ad creative places this split test after organic pre-testsPosting creative ideas as unpaid organic content to see whether the feed responds at all, before money is spent on rotation or a controlled split test. and rotation inside an ad group, for the questions worth proving.
| Design element | Choice | Reason |
|---|---|---|
| Method | Platform split test with separate audience groups | Stops delivery skew between versions |
| Variants | Brand-logo opening versus problem-led opening | One change: the opening seconds |
| Edit after the opening | Frame-for-frame identical | Credits any difference to the hook |
| Budget | Equal per arm | Comparable delivery |
| Targeting, placements, optimisation event | Identical | Only the creative varies |
| Duration | Whole weeks, past the platform's early delivery period | Avoids reading noisy early results |
Set the duration in whole weeks and long enough to pass the early, unstable phase of delivery. The creative testing guide describes how delivery settles over the first days of a new ad, and a hook test should be read only once that has happened.
Sample size reasoning
The quantity that drives this test is acquisitions per arm, not impressions. Impressions are plentiful, but cost per acquisition is only stable once each arm has gathered enough conversions. The inputs are your current cost per acquisition, how much of a reduction would justify changing your creative approach, and the confidence you require. The sample size and significance guide explains how those combine, and the platforms' split test tools usually report their own estimate of confidence, which is worth reading alongside your own.
Translate the needed acquisitions into budget and time using your current cost per acquisition. If that is more than the test can spend, test on a higher-funnel conversion event that occurs more often, and accept that the result then speaks to that event, not to sales.
Reading the results
Read when the planned duration and acquisitions per arm have both been reached. Compare cost per acquisition between arms and check whether the difference exceeds the noise your method allows for. Then look at the guardrails. A lower cost per acquisition with steady post-click conversion rate and similar frequency is a clean win for the problem-led opening.
Use the diagnostics to explain the result. If the problem-led opening kept more viewers past the opening and had a higher CTRShare of impressions that result in a click., the mechanism in the hypothesis held. If it held attention but cost per acquisition did not improve, the hook attracted interest the offer could not convert; the next test belongs on the offer or the landing page. A flat result is common and useful: it suggests the opening was not the bottleneck. Record the result and the edit files so a future team can see exactly what was tested.
A running shoe brand tests a problem-led opening
Stridewell, an invented running shoe brand, runs a video ad that opens with its logo over a slow-motion shot of a shoe. The challenger opens with a runner wincing and the caption "Shin splints after every long run?", then cuts to the same footage. A platform split test runs for three weeks with £4,000 per arm. The problem-led version records 210 purchases against 160 for the logo opening, a cost per acquisition of about £19 against £25. Post-click conversion rate is similar in both arms and frequency stays close. The brand moves problem-led openings into its creative guidelines.
Pitfalls
Delivery skew is the pitfall this design exists to prevent. When two ads share an ad set, the system quickly favours one, and the "loser" may never have reached a comparable audience. Always use a split test with separate audience groups for a learning test.
Early winners are the second. In the first days, cost per acquisition swings widely, and a version that looks better on day two can look worse by the end of the week. Read only at the planned point. The third pitfall is treating attention metrics as the goal. Thumb-stop rate and watch time are easy to move with shock openings that attract the wrong audience; they are diagnostics, not outcomes. Finally, watch for platform features that alter creative automatically, such as automatic enhancements or cropping. If they are on for one arm, they must be on for both, or switched off for the test.
On LinkedIn, where audiences are small and impressions costly, the guide to testing LinkedIn ad creative explains when a split test like this one is worth its budget and when rotation or people's own posts can answer the question more cheaply.
Common mistakes
Creative tests attract more opinion than most, so a clean design is the only defence against arguments afterwards. The same discipline applies to search ads in the search ad asset test, and the experiment library lists the designs for budgets and audiences.
- Testing two different ads and calling it a hook test. Only the opening may differ.
- Running both versions in one ad set. Delivery will favour one and the comparison breaks.
- Reading in the first days. Early cost per acquisition is noisy and misleading.
- Optimising for watch time. Attention that does not convert is not a win.
- Unequal budgets. Different spend levels change delivery and cost on their own.
- Leaving automatic creative changes on for one arm. That adds a second, invisible variable.
