Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Experiment Library

SEO Title Tag Test

A design for testing title tag changes across a template of similar pages: a random page-group split, clicks per page, position and rewrite guardrails.
Experimentadvanced7 min readUpdated

Hypothesis

If the title tags on a random half of a large set of similar pages are rewritten to state the searched benefit, organic clicks per page will rise relative to the unchanged half, because a clearer title earns more clicks at the same position. The decision is whether to roll a new title tag pattern out across a page template, such as product, category or location pages, or to keep the current pattern.

A pattern is the right thing to test, not a single title. One page's clicks move for reasons that have nothing to do with its title: a competitor's new page, a seasonal query, a change in the results layout. A pattern applied to many comparable pages averages that noise away. The title tags and meta descriptions guide covers how to write titles for generated page types and sketches the group comparison; this design turns that sketch into a plan with a fixed split, a pre-period and a decision rule written in advance. Record the current pattern, the new one and the pages in scope in the experiment hypothesis template.

Metrics and guardrails

The primary metric is organic clicks per page, read as the change in the variant group relative to the change in the control group over the same weeks. Comparing changes rather than levels removes the difference the two groups had before the test; comparing groups over the same weeks removes everything that happened to both, from algorithm updates to seasonality. Search Console's performance report gives clicks, impressions, click-through rate and average position by page, and the Google Search Console guide explains how to filter and export it.

Three guardrails keep the result honest. Average position checks that the change moved click-through (CTR) rather than rankings; if position moved, the test is no longer reading the title's pull at the same spot on the SERP. The share of titles rewritten by Google checks that searchers actually saw the new pattern: Google Search Central's documentation on title links explains that the title element is one of several sources it may use, so sample variant pages in search results after the recrawl and record what is shown. Organic conversions per page check that the extra clicks are the right ones, since a title can earn clicks by promising something the page does not deliver. The SEO KPIs and reporting guide covers how these sit in a regular report.

Design

Start from one template and list every page that has steady organic traffic over a recent period. Exclude pages being changed for other reasons during the test. Split the list at random into a variant group and a control group; if traffic is very uneven, sort pages into traffic bands first and randomise within each band so both groups carry a similar mix. Run a pre-period with no change and confirm that the two groups' clicks move together. Then apply the new pattern to the variant group only, request recrawling where the site's tooling allows, and read both groups over the same post-period. The academy exercise on designing a title test across a tyre retailer's vehicle pages practises this design with clicks per page as the metric, position and rewrite guardrails and a decision rule written before launch.

Design elementChoiceReason
Unit of randomisationPage, within one templateSearchers cannot be randomised; pages can
GroupsVariant with the new pattern, control unchangedIsolates the title change
BalanceRandom assignment within traffic bandsStops a few large pages dominating one group
Pre-periodBoth groups unchanged; check they move togetherConfirms the groups are comparable
Held constantContent, internal links, structured data, templates in both groupsOnly the title differs
Primary metricChange in clicks per page, variant relative to controlRemoves shared shocks and starting differences

Sample size reasoning

Visitors cannot be randomised in organic search, so the unit is the page and the sample is the number of similar pages with steady traffic. The noise is how much clicks per page wander from week to week, and the fewer pages a template has, the larger the change must be before it can be told apart from that wander. The logic is the one the incrementality testing guide sets out for designs with few, noisy units: decide the smallest change that would justify rolling the pattern out, check from the pre-period how much the groups' difference wanders, and only run the test if the expected change could stand out from it. The CTR calculator for organic search helps translate a click-through difference at a given position into clicks, which is how the smallest change worth acting on is usually set. If the template is too small, test a bolder pattern or pool comparable templates that share it. The academy checkpoint on measuring and testing SEO changes tests this sample size reasoning beside comparison pages, confounders and internal linking tests.

Duration

Run a pre-period long enough to confirm the groups move together, then several weeks after the change has been recrawled, with no other template changes in either group. The post-period starts when the new titles are in the index, not when they were published; until then, the variant group is still showing the old titles. Avoid starting in the middle of a known seasonal shift for the template's queries, and freeze other on-page work on both groups until the read.

Reading the results and the decision rule

Read once at the planned end. Compute each group's change in clicks per page from pre-period to post-period, then the difference between those changes, with an interval built from how much that difference wandered in the pre-period. Then read position, the sampled rewrite share and conversions. The rule, written before launch:

ResultDecision
Variant clicks rise relative to control, position and conversions holdRoll the pattern out across the template
Variant clicks rise but position moved tooTreat as inconclusive; the title may have changed ranking, not click-through
Google rewrites most variant titlesRevert and diagnose before testing another pattern
Variant clicks fall relative to controlRevert the variant group to the old pattern
No reliable differenceKeep the current pattern; test a bolder change or a larger template

Pitfalls

Recrawl timing is the first pitfall. Pages are recrawled at different speeds, so some variant pages show the new title weeks before others, and a post-period that starts on the publication date mixes old and new titles. Check a sample of variant pages in the index and start the post-period once most show the change.

Algorithm updates hitting both groups are the second. A core update during the test moves clicks on many pages, which is exactly why the comparison is between groups over the same weeks rather than before and after. An update that affects the template unevenly can still distort the result, so note its date and check whether the groups' difference jumped on that day. The third is title rewrites hiding the change. If Google displays a heading or anchor text instead of the new title, the variant group's searchers never saw the pattern. The troubleshooting page on titles rewritten in search results explains how to confirm a rewrite and what causes it. When titles have already been changed without a random split, the tutorial on measuring an SEO change after the fact compares the changed pages with a similar set left alone over matched windows and states the result with its uncertainty.

Illustrative example — not a real result

A tile retailer tests benefit-led category titles

Corriemoor Tiles, an invented tile retailer, has 640 category pages built from one template. It drops 40 pages scheduled for redesign and splits the remaining 600, within traffic bands, into 300 variant and 300 control pages. Over an eight-week pre-period, the groups' weekly clicks per page track each other closely. The variant pattern replaces "Grey Porcelain Tiles | Corriemoor" with "Grey Porcelain Tiles: Frost-Proof, Free Samples | Corriemoor". After recrawling, the team reads six weeks. Variant clicks per page rise 9% against 2% in control, a relative difference of about 7% with an interval of roughly 3% to 11%. Average position is unchanged, a sample of 50 variant results shows 6 rewritten titles, and conversions per page rise in line with clicks. Corriemoor rolls the pattern out to every category page.

Common mistakes

Title tests fail most often on timing and on reading the wrong numbers. The same page-group split carries over to the internal linking test, and the experiment library lists the other designs.

  • Testing one page. A single page's clicks move for too many reasons to read a title change.
  • Comparing before and after without a control. Seasonality and updates masquerade as effects.
  • Starting the post-period at publication. The new titles are not in the index yet.
  • Ignoring position. A ranking change is not a click-through result.
  • Not checking what Google displays. Rewritten titles mean the pattern was never seen.
  • Changing other elements on the variant pages. Then the test reads the bundle, not the title.

Frequently asked questions

Why split pages instead of visitors?

Search engines see one version of each page, and every searcher sees the same title link for a given page. There is no way to show half the searchers a different title, so the experiment randomises pages instead: comparable pages are assigned to a changed group and an unchanged group, and the groups are compared over the same weeks.

Can this design test meta descriptions too?

Yes, with the same split, metric and guardrails. The signal is usually weaker, because search engines often show a snippet drawn from the page rather than the meta description, so check how often the description is actually displayed before spending a test on it.

What if Google rewrites the new titles?

Then the test is not testing the titles searchers saw. That is why the share of rewritten titles is a guardrail: sample the variant pages in search results after the recrawl and record what is displayed. If most variant titles are rewritten, revert and diagnose why before testing another pattern.

Terms used on this page

Tools to try

Frameworks

Where this page is used

Part of these learning paths

Also related

Sources

  1. Influencing your title links in search results — Google Search Central
  2. Performance report (Search results) — Google Search Console Help

Related