Hypothesis
If a random half of articles with declining organic traffic are substantively refreshed, their organic clicks will recover relative to the unrefreshed half, because updated, more complete pages better satisfy current searches. The decision is whether to fund a programme of refreshing decaying articles, or to put the hours into new content.
Most teams already believe refreshes work, and most have a story of one article that came back after an update. The test exists because the story cannot tell the refresh apart from what would have happened anyway. The troubleshooting page on old content losing traffic diagnoses why an archive decays and which pages to fix first; this design asks whether the fix pays at the scale of a programme. Record the decay rule, the brief, the hours budget and the reading in the experiment hypothesis template before any article is touched.
Metrics and guardrails
The primary metric is organic clicks per article, read as the change in the refreshed group relative to the change in the control group over the same weeks. Search Console's performance report gives clicks, impressions and position by page, and the Google Search Console guide explains how to export them for two fixed lists of pages.
Three guardrails keep the refresh honest. Rankings for queries the articles already won check that the update did not lose what the page was still earning; a rewrite that chases new queries can drop old ones, so record each article's leading queries before the test and read their position afterwards. Organic conversions per article check that recovered traffic is worth something, using the content-to-outcome definitions in the guide to measuring content performance. Editorial hours per refreshed article are the cost side of the decision: log them as the work is done, because the rule compares the refresh's return with what the same hours would earn as new content.
Design
List decaying articles by a fixed rule written before anyone looks at the pages: for example, articles whose organic clicks fell steadily over a stated period while the topic still has search demand. Exclude articles that are being merged, redirected or retired, and articles whose decline started on one day, which points to a technical cause rather than decay. Split the list at random into a refresh group and a control group, within traffic bands if traffic is uneven. Refresh every variant article to one brief; leave every control article untouched.
| Design element | Choice | Reason |
|---|---|---|
| Unit of randomisation | Article, from a list built by a fixed decay rule | Pages cannot be randomised by visitor |
| Groups | Refreshed to one brief, against untouched | Separates the refresh from natural recovery |
| Brief | One refresh brief applied to every variant article | Makes the treatment consistent and repeatable |
| Addresses | Every article keeps its URL in both groups | Avoids testing a migration |
| Held constant | Internal links, templates and navigation for both groups | Only the content differs |
| Primary metric | Change in clicks per article, refreshed relative to control | Removes shared shocks and recovery |
The brief does the work. Use the structure from the SEO content briefs guide: the current Search IntentThe goal behind a search query: to learn, to find a site, to compare or to act. behind the article's leading queries, what the pages now ranking above it cover, the facts and examples to update, and the sections to add or remove. How each refreshed article shows its updated date and change note is a house rule set once, as the guide to updating and correcting published content explains, so every refreshed article follows the same rule. Google Search Central's guidance on helpful, reliable, people-first content offers questions about originality, completeness and trust that a refresh brief can use as a checklist, and it is the same ground the E-E-A-TExperience, Expertise, Authoritativeness, Trust. concept covers. If articles sit in clusters around a Content PillarOne of a small set of recurring themes that a brand's content is planned around., keep the pillar out of both groups so that changes to it do not reach the control articles through links.
Sample size reasoning
The sample is the number of decaying articles; selecting them because they declined means some would recover anyway, which is exactly why the untouched control group is needed. Each article contributes a noisy weekly series, and the fewer articles there are, the larger the recovery must be before it stands out from the wander between groups. The incrementality testing guide covers reading a difference between few, noisy groups and setting the smallest change worth acting on before the test. The SEO traffic estimator helps translate a plausible position recovery into clicks, which is a practical way to set that smallest change. If the decaying list is short, pool articles from several sections under the same rule, or accept that only a large recovery will be visible.
Duration
Run a pre-period, then several weeks to a few months after the refreshed pages are recrawled, because ranking changes settle slowly. Start the post-period when most refreshed articles show the new content in the index, not on the day editors finished. Avoid launching just before a known seasonal shift in the topics, and freeze other work on both groups, including new internal links from fresh articles, until the read.
Reading the results and the decision rule
Read once at the planned end. Compute each group's change in clicks per article from pre-period to post-period, the difference between them, and an interval built from the pre-period wander. Then read the guardrails, and convert the difference into clicks per editorial hour. The rule, written before launch:
| Result | Decision |
|---|---|
| Refreshed group recovers relative to control at a cost per click below new content's expected return | Fund the refresh programme under the same brief |
| Both groups recover similarly | The decline was partly temporary; refresh only articles with a clear content gap |
| Refreshed group recovers but loses rankings it already held | Revise the brief to protect existing queries before scaling |
| No reliable recovery relative to control | Prune or merge decayed articles instead of refreshing them |
The monthly SEO content operations playbook shows where a positive result goes next: into the refresh queue, with the decay rule as the entry criterion and the brief as the standard.
Pitfalls
Regression to the mean without a control is the defining pitfall. Articles picked at the bottom of a decline include some that dipped temporarily, and they drift back whatever is done to them. A refresh programme read without a control group takes credit for that drift and is funded on a result that would have happened anyway.
Changing URLs during a refresh is the second. A new address means a redirect, a period of re-evaluation and the loss of some signals tied to the old address, so the test measures a migration bundled with an update. Keep every address. The third is refreshing only the best candidates. If editors choose which decaying articles to refresh, they pick the ones with the most obvious gaps and the best prospects, and the comparison with the remaining articles is biased before the first word is written. Randomise first, then refresh every article that landed in the variant group, including the unpromising ones. Where a refresh has already shipped without randomisation, the tutorial on measuring an SEO change gives the weaker but still useful after-the-fact read against articles that were left unchanged.
A sailing school tests refreshing its guides
Hollinsby Sailing School, an invented sailing school with a large blog, lists 120 articles whose organic clicks fell steadily over the past year while their topics still had search demand. It splits them into two groups of 60. Editors refresh the variant group to one brief at an average of 3.5 hours per article. Ten weeks after recrawling, refreshed articles' clicks per article are up 22% from the pre-period and the control group's up 8%, a relative difference of about 13% with an interval of roughly 4% to 22%. Rankings for the articles' existing leading queries hold, and sign-ups for taster sessions from the refreshed articles rise in line with clicks. At the logged hours, the refreshes earn more clicks per hour than the school's recent new articles, so Hollinsby funds a quarterly refresh queue.
Common mistakes
Refresh tests fail when the control is missing or the treatment changes from page to page. The internal linking test uses the same page-group split for links, and the experiment library lists every design.
- No control group. Natural recovery is credited to the refresh.
- Letting editors choose which articles to refresh. Selection bias decides the result.
- Changing URLs. The test then measures a migration.
- Refreshing to different standards. One brief for every variant article.
- Reading too soon. Rankings settle over weeks to months after recrawling.
- Ignoring the hours. The decision compares the refresh with new content, not with nothing.
