Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Experiment Library

Content Refresh Test

A design for testing whether refreshing decaying articles recovers search traffic: refresh a random half and compare it with the untouched half.
Experimentintermediate7 min readUpdated

Hypothesis

If a random half of articles with declining organic traffic are substantively refreshed, their organic clicks will recover relative to the unrefreshed half, because updated, more complete pages better satisfy current searches. The decision is whether to fund a programme of refreshing decaying articles, or to put the hours into new content.

Most teams already believe refreshes work, and most have a story of one article that came back after an update. The test exists because the story cannot tell the refresh apart from what would have happened anyway. The troubleshooting page on old content losing traffic diagnoses why an archive decays and which pages to fix first; this design asks whether the fix pays at the scale of a programme. Record the decay rule, the brief, the hours budget and the reading in the experiment hypothesis template before any article is touched.

Metrics and guardrails

The primary metric is organic clicks per article, read as the change in the refreshed group relative to the change in the control group over the same weeks. Search Console's performance report gives clicks, impressions and position by page, and the Google Search Console guide explains how to export them for two fixed lists of pages.

Three guardrails keep the refresh honest. Rankings for queries the articles already won check that the update did not lose what the page was still earning; a rewrite that chases new queries can drop old ones, so record each article's leading queries before the test and read their position afterwards. Organic conversions per article check that recovered traffic is worth something, using the content-to-outcome definitions in the guide to measuring content performance. Editorial hours per refreshed article are the cost side of the decision: log them as the work is done, because the rule compares the refresh's return with what the same hours would earn as new content.

Design

List decaying articles by a fixed rule written before anyone looks at the pages: for example, articles whose organic clicks fell steadily over a stated period while the topic still has search demand. Exclude articles that are being merged, redirected or retired, and articles whose decline started on one day, which points to a technical cause rather than decay. Split the list at random into a refresh group and a control group, within traffic bands if traffic is uneven. Refresh every variant article to one brief; leave every control article untouched.

Design elementChoiceReason
Unit of randomisationArticle, from a list built by a fixed decay rulePages cannot be randomised by visitor
GroupsRefreshed to one brief, against untouchedSeparates the refresh from natural recovery
BriefOne refresh brief applied to every variant articleMakes the treatment consistent and repeatable
AddressesEvery article keeps its URL in both groupsAvoids testing a migration
Held constantInternal links, templates and navigation for both groupsOnly the content differs
Primary metricChange in clicks per article, refreshed relative to controlRemoves shared shocks and recovery

The brief does the work. Use the structure from the SEO content briefs guide: the current Search Intent behind the article's leading queries, what the pages now ranking above it cover, the facts and examples to update, and the sections to add or remove. How each refreshed article shows its updated date and change note is a house rule set once, as the guide to updating and correcting published content explains, so every refreshed article follows the same rule. Google Search Central's guidance on helpful, reliable, people-first content offers questions about originality, completeness and trust that a refresh brief can use as a checklist, and it is the same ground the E-E-A-T concept covers. If articles sit in clusters around a Content Pillar, keep the pillar out of both groups so that changes to it do not reach the control articles through links.

Sample size reasoning

The sample is the number of decaying articles; selecting them because they declined means some would recover anyway, which is exactly why the untouched control group is needed. Each article contributes a noisy weekly series, and the fewer articles there are, the larger the recovery must be before it stands out from the wander between groups. The incrementality testing guide covers reading a difference between few, noisy groups and setting the smallest change worth acting on before the test. The SEO traffic estimator helps translate a plausible position recovery into clicks, which is a practical way to set that smallest change. If the decaying list is short, pool articles from several sections under the same rule, or accept that only a large recovery will be visible.

Duration

Run a pre-period, then several weeks to a few months after the refreshed pages are recrawled, because ranking changes settle slowly. Start the post-period when most refreshed articles show the new content in the index, not on the day editors finished. Avoid launching just before a known seasonal shift in the topics, and freeze other work on both groups, including new internal links from fresh articles, until the read.

Reading the results and the decision rule

Read once at the planned end. Compute each group's change in clicks per article from pre-period to post-period, the difference between them, and an interval built from the pre-period wander. Then read the guardrails, and convert the difference into clicks per editorial hour. The rule, written before launch:

ResultDecision
Refreshed group recovers relative to control at a cost per click below new content's expected returnFund the refresh programme under the same brief
Both groups recover similarlyThe decline was partly temporary; refresh only articles with a clear content gap
Refreshed group recovers but loses rankings it already heldRevise the brief to protect existing queries before scaling
No reliable recovery relative to controlPrune or merge decayed articles instead of refreshing them

The monthly SEO content operations playbook shows where a positive result goes next: into the refresh queue, with the decay rule as the entry criterion and the brief as the standard.

Pitfalls

Regression to the mean without a control is the defining pitfall. Articles picked at the bottom of a decline include some that dipped temporarily, and they drift back whatever is done to them. A refresh programme read without a control group takes credit for that drift and is funded on a result that would have happened anyway.

Changing URLs during a refresh is the second. A new address means a redirect, a period of re-evaluation and the loss of some signals tied to the old address, so the test measures a migration bundled with an update. Keep every address. The third is refreshing only the best candidates. If editors choose which decaying articles to refresh, they pick the ones with the most obvious gaps and the best prospects, and the comparison with the remaining articles is biased before the first word is written. Randomise first, then refresh every article that landed in the variant group, including the unpromising ones. Where a refresh has already shipped without randomisation, the tutorial on measuring an SEO change gives the weaker but still useful after-the-fact read against articles that were left unchanged.

Illustrative example — not a real result

A sailing school tests refreshing its guides

Hollinsby Sailing School, an invented sailing school with a large blog, lists 120 articles whose organic clicks fell steadily over the past year while their topics still had search demand. It splits them into two groups of 60. Editors refresh the variant group to one brief at an average of 3.5 hours per article. Ten weeks after recrawling, refreshed articles' clicks per article are up 22% from the pre-period and the control group's up 8%, a relative difference of about 13% with an interval of roughly 4% to 22%. Rankings for the articles' existing leading queries hold, and sign-ups for taster sessions from the refreshed articles rise in line with clicks. At the logged hours, the refreshes earn more clicks per hour than the school's recent new articles, so Hollinsby funds a quarterly refresh queue.

Common mistakes

Refresh tests fail when the control is missing or the treatment changes from page to page. The internal linking test uses the same page-group split for links, and the experiment library lists every design.

  • No control group. Natural recovery is credited to the refresh.
  • Letting editors choose which articles to refresh. Selection bias decides the result.
  • Changing URLs. The test then measures a migration.
  • Refreshing to different standards. One brief for every variant article.
  • Reading too soon. Rankings settle over weeks to months after recrawling.
  • Ignoring the hours. The decision compares the refresh with new content, not with nothing.

Frequently asked questions

Why leave half the decaying articles untouched?

Because articles chosen for declining will partly recover on their own. Some declines are temporary dips, and a group selected at its low point tends to drift back towards its usual level. Without an untouched group, that natural recovery is credited to the refresh, and the programme looks better than it is.

What counts as a substantive refresh?

One that changes what the page offers the reader: facts and examples brought up to date, sections added for questions the page no longer answers, outdated sections removed, and the structure rebuilt around the current search intent. Changing the date, a few words or the title alone is not a refresh and should not be tested as one.

Should refreshed articles keep their URLs?

Yes. Changing the address during a refresh adds a redirect and a period of re-evaluation to the treatment, so the test would measure a migration as well as an update. Keep every article at its existing address in both groups.

Terms used on this page

Tools to try

Frameworks

Where this page is used

Also related

Sources

  1. Creating helpful, reliable, people-first content — Google Search Central
  2. Performance report (Search results) — Google Search Console Help

Related