Hypothesis
If the title tags on a random half of a large set of similar pages are rewritten to state the searched benefit, organic clicks per page will rise relative to the unchanged half, because a clearer title earns more clicks at the same position. The decision is whether to roll a new title tag pattern out across a page template, such as product, category or location pages, or to keep the current pattern.
A pattern is the right thing to test, not a single title. One page's clicks move for reasons that have nothing to do with its title: a competitor's new page, a seasonal query, a change in the results layout. A pattern applied to many comparable pages averages that noise away. The title tags and meta descriptions guide covers how to write titles for generated page types and sketches the group comparison; this design turns that sketch into a plan with a fixed split, a pre-period and a decision rule written in advance. Record the current pattern, the new one and the pages in scope in the experiment hypothesis template.
Metrics and guardrails
The primary metric is organic clicks per page, read as the change in the variant group relative to the change in the control group over the same weeks. Comparing changes rather than levels removes the difference the two groups had before the test; comparing groups over the same weeks removes everything that happened to both, from algorithm updates to seasonality. Search Console's performance report gives clicks, impressions, click-through rate and average position by page, and the Google Search Console guide explains how to filter and export it.
Three guardrails keep the result honest. Average position checks that the change moved click-through (CTRShare of impressions that result in a click.) rather than rankings; if position moved, the test is no longer reading the title's pull at the same spot on the SERPThe page of results a search engine shows.. The share of titles rewritten by Google checks that searchers actually saw the new pattern: Google Search Central's documentation on title links explains that the title element is one of several sources it may use, so sample variant pages in search results after the recrawl and record what is shown. Organic conversions per page check that the extra clicks are the right ones, since a title can earn clicks by promising something the page does not deliver. The SEO KPIs and reporting guide covers how these sit in a regular report.
Design
Start from one template and list every page that has steady organic traffic over a recent period. Exclude pages being changed for other reasons during the test. Split the list at random into a variant group and a control group; if traffic is very uneven, sort pages into traffic bands first and randomise within each band so both groups carry a similar mix. Run a pre-period with no change and confirm that the two groups' clicks move together. Then apply the new pattern to the variant group only, request recrawling where the site's tooling allows, and read both groups over the same post-period. The academy exercise on designing a title test across a tyre retailer's vehicle pages practises this design with clicks per page as the metric, position and rewrite guardrails and a decision rule written before launch.
| Design element | Choice | Reason |
|---|---|---|
| Unit of randomisation | Page, within one template | Searchers cannot be randomised; pages can |
| Groups | Variant with the new pattern, control unchanged | Isolates the title change |
| Balance | Random assignment within traffic bands | Stops a few large pages dominating one group |
| Pre-period | Both groups unchanged; check they move together | Confirms the groups are comparable |
| Held constant | Content, internal links, structured data, templates in both groups | Only the title differs |
| Primary metric | Change in clicks per page, variant relative to control | Removes shared shocks and starting differences |
Sample size reasoning
Visitors cannot be randomised in organic search, so the unit is the page and the sample is the number of similar pages with steady traffic. The noise is how much clicks per page wander from week to week, and the fewer pages a template has, the larger the change must be before it can be told apart from that wander. The logic is the one the incrementality testing guide sets out for designs with few, noisy units: decide the smallest change that would justify rolling the pattern out, check from the pre-period how much the groups' difference wanders, and only run the test if the expected change could stand out from it. The CTR calculator for organic search helps translate a click-through difference at a given position into clicks, which is how the smallest change worth acting on is usually set. If the template is too small, test a bolder pattern or pool comparable templates that share it. The academy checkpoint on measuring and testing SEO changes tests this sample size reasoning beside comparison pages, confounders and internal linking tests.
Duration
Run a pre-period long enough to confirm the groups move together, then several weeks after the change has been recrawled, with no other template changes in either group. The post-period starts when the new titles are in the index, not when they were published; until then, the variant group is still showing the old titles. Avoid starting in the middle of a known seasonal shift for the template's queries, and freeze other on-page work on both groups until the read.
Reading the results and the decision rule
Read once at the planned end. Compute each group's change in clicks per page from pre-period to post-period, then the difference between those changes, with an interval built from how much that difference wandered in the pre-period. Then read position, the sampled rewrite share and conversions. The rule, written before launch:
| Result | Decision |
|---|---|
| Variant clicks rise relative to control, position and conversions hold | Roll the pattern out across the template |
| Variant clicks rise but position moved too | Treat as inconclusive; the title may have changed ranking, not click-through |
| Google rewrites most variant titles | Revert and diagnose before testing another pattern |
| Variant clicks fall relative to control | Revert the variant group to the old pattern |
| No reliable difference | Keep the current pattern; test a bolder change or a larger template |
Pitfalls
Recrawl timing is the first pitfall. Pages are recrawled at different speeds, so some variant pages show the new title weeks before others, and a post-period that starts on the publication date mixes old and new titles. Check a sample of variant pages in the index and start the post-period once most show the change.
Algorithm updates hitting both groups are the second. A core update during the test moves clicks on many pages, which is exactly why the comparison is between groups over the same weeks rather than before and after. An update that affects the template unevenly can still distort the result, so note its date and check whether the groups' difference jumped on that day. The third is title rewrites hiding the change. If Google displays a heading or anchor text instead of the new title, the variant group's searchers never saw the pattern. The troubleshooting page on titles rewritten in search results explains how to confirm a rewrite and what causes it. When titles have already been changed without a random split, the tutorial on measuring an SEO change after the fact compares the changed pages with a similar set left alone over matched windows and states the result with its uncertainty.
A tile retailer tests benefit-led category titles
Corriemoor Tiles, an invented tile retailer, has 640 category pages built from one template. It drops 40 pages scheduled for redesign and splits the remaining 600, within traffic bands, into 300 variant and 300 control pages. Over an eight-week pre-period, the groups' weekly clicks per page track each other closely. The variant pattern replaces "Grey Porcelain Tiles | Corriemoor" with "Grey Porcelain Tiles: Frost-Proof, Free Samples | Corriemoor". After recrawling, the team reads six weeks. Variant clicks per page rise 9% against 2% in control, a relative difference of about 7% with an interval of roughly 3% to 11%. Average position is unchanged, a sample of 50 variant results shows 6 rewritten titles, and conversions per page rise in line with clicks. Corriemoor rolls the pattern out to every category page.
Common mistakes
Title tests fail most often on timing and on reading the wrong numbers. The same page-group split carries over to the internal linking test, and the experiment library lists the other designs.
- Testing one page. A single page's clicks move for too many reasons to read a title change.
- Comparing before and after without a control. Seasonality and updates masquerade as effects.
- Starting the post-period at publication. The new titles are not in the index yet.
- Ignoring position. A ranking change is not a click-through result.
- Not checking what Google displays. Rewritten titles mean the pattern was never seen.
- Changing other elements on the variant pages. Then the test reads the bundle, not the title.
