What this checkpoint covers
Drabbelon Guitars is an invented online shop selling guitars, amplifiers and effects pedals, with a product page template shared across its ranges and a set of buying guides. Its agency, merchandising manager and content lead each have a view on what the last title rewrite did and what to try next. The questions apply the tutorial on measuring an SEO change after the fact, the SEO title tag test design and the internal linking test design.
Drabbelon Guitars: templates, changes and proposals
| Template | Pages | Pages with steady organic clicks | Recent or planned change |
|---|---|---|---|
| Electric guitar model pages | 1,400 | 1,150 | Title test proposed |
| Acoustic guitar model pages | 900 | 720 | Title test finished last month |
| Amplifier pages | 380 | 300 | Titles rewritten on 4 March |
| Effects pedal pages | 620 | 410 | Title test proposed for December |
| Buying guides | 160 | 90 | Linking work proposed |
- Amplifier rewrite: the agency's chart shows amplifier clicks in April 22% above February. A site-wide sale ran through April, a search update was announced on 18 March, and a filter menu was added to the amplifier category pages on 9 April. Effects pedal pages share the product template and were untouched.
- Electric proposal: new titles on the 200 best-selling models, compared with the other 950.
- Acoustic test: 360 variant and 360 control pages split at random within traffic bands, read six weeks after recrawl. Clicks per page: variant up 9%, control up 1%, a relative difference of about 8% with an interval of roughly 1% to 15%. Average position: variant 6.2 to 5.5, control 6.3 to 6.3. Sampled results: 11 of 40 variant pages show a rewritten title. Organic conversions per page flat in both groups.
- Buying guides: 70 have three or fewer inbound links and 10 of those are due for rewrite. The content lead wants links to all 70 from the footer and from any product page with space, read on clicks after a month. A new menu ships in six weeks.
- Pedal proposal: a two-week test from the publication date in December, the gifting peak. Between two random halves of pedal pages, the weekly difference wandered by about ±6% in the pre-period; the smallest change worth acting on is 3%.
Task
- Write a decision and a reason for every question; an answer counts only if the reason holds.
- Then read the explanations, and reread the linked section behind each miss.
Questions
- The agency presents its April against February chart as proof the amplifier titles worked. What do you tell the owner?
- You are asked to judge the amplifier rewrite properly. Which pages do you compare with, over which windows, and what do you check before stating a result?
- The merchandising manager wants the new electric guitar titles on the best sellers. What design do you propose instead?
- For the electric test, the agency plans to read the site's overall click-through rate. What do you measure, and what guards the reading?
- How would you turn the content lead's linking proposal into a test the business can scale from?
- Should the pedal test run as proposed?
- How do you report the acoustic result?
Answers explained
- It shows movement, not cause. A sale, a search update and a filter release all landed in the same weeks, and February and April are different points in the year. A before-and-after line credits all of it to the titles. See why before-and-after comparisons mislead.
- Pedal pages over matched windows. They share the template and were untouched, so they can act as a Holdout GroupA randomly chosen part of an audience kept from exposure to measure what exposure caused. chosen afterwards, if their clicks moved with the amplifier pages before the change and none share queries or gained links from them. Start the post window once sampled amplifier pages show a crawl after the release, match lengths and weekdays, and either end it before the filter menu or name that release as a confounder the comparison cannot remove. Check the update week and sample which titles searchers saw. See comparison pages, windows and confounders.
- A random split within traffic bands. A group chosen for its sales is not comparable with the rest by design, so whatever already sets it apart enters the result, and a few large pages would dominate it. Exclude pages due for other changes, randomise the remainder, keep the control as the holdout group and confirm in a pre-period that both groups move together. See the title tag test design.
- Clicks per page, variant against control. A site-wide rate mixes every template and every season. The change in clicks per page relative to the control is the metric, with average position, the share of titles Google rewrote and organic conversions per page as guardrails, all written down before launch. See metrics and guardrails.
- Split the targets, fix the rule, freeze navigation. Drop the guides due for rewrite and split the rest at random. Only variant guides gain contextual links, from product pages whose content discusses the guide's topic, under one rule. Footer links reach both groups, as will the new menu, so the test runs before or after it. Read impressions as well as clicks, and guard the source pages' clicks and the targets' indexing status. Each Internal LinkA link from one page to another page on the same website. added must be crawlable. See the internal linking test design.
- No. The post-period starts when new titles are indexed, not at publication, and a gifting peak can move the two halves differently. With pedal pages wandering far more than the change worth acting on, Sample SizeThe number of users or sessions an experiment needs to detect a given effect reliably. is the real limit: test a bolder pattern, or pool templates that share it, and run several weeks outside the peak. See sample size and duration.
- As inconclusive, not as a win. Clicks rose against the control, but position moved too, so the titles may have changed ranking rather than click-through, and a sizeable share of variant titles were not shown as written. Report the range, both caveats and flat conversions. The interval that stands in for Statistical SignificanceA test result unlikely enough under 'no real difference' to pass a threshold chosen in advance. here is built from how much the groups' difference wandered in the pre-period, because pages, not visitors, were randomised. See reading a title test's results and stating the result honestly.
Where to go next
Misses on the first two questions call for the tyre retailer title exercise again; misses on the rest call for the two experiment designs. The advanced SEO level covers the rankings drop diagnosis these tests sit beside, and the next module's practice is the breed page programme exercise.
