Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Experiment Library

Shopping Feed Title Test

A product-group experiment for rewriting Shopping feed titles: half a category retitled through a supplemental feed, read on conversions per product.
Experimentadvanced9 min readUpdated

Hypothesis

If a random half of the products in a category get feed titles rewritten to the category's search-matched pattern through a supplemental feed, those products will win more relevant impressions and conversions than the untouched half, because the title is the main text matched against searches, unless the campaign simply moves budget away from the untouched products. The decision behind it is concrete: rewrite the product feed titles in a category to a search-matched pattern, or keep the current titles.

The guide to product feeds and Merchant Center sets the title patterns and explains why they matter, stating from Merchant Center's product data specification that the title is the main matching text, that truncation in the ad happens from the end and that promotional text is not allowed in titles. Its own illustrative retitling is read before and after. That reading is the usual one, and it cannot separate the new titles from the season, competitors and the campaign's own changes over the same weeks. This design keeps a random half of the category on its old titles so that the comparison lives through the same weeks.

Write the pattern, the category, the halves and the limits on each guardrail into the experiment hypothesis template before the supplemental feed is uploaded. The experimentation loop puts that step before the build, which matters here because a feed change is quick to make and tempting to roll out on a hunch.

Metrics and guardrails

The primary metric is conversions per product, or conversion value per product where products differ widely in price, in the treated half against the control half, each relative to its own pre-period. Take each half's conversions per product in the post-period, divide by the same half's figure in the pre-period, and compare the two ratios. Reading each half against its own past removes the starting difference between them; reading the halves against each other removes anything that moved the whole category. The measuring SEO changes guide sets out the same logic for pages, and it transfers to products unchanged.

Report impressions, CTR and CVR per half as diagnostics. A title that matches more relevant searches should show more impressions on specific queries and a higher click-through on them; the CTR calculator turns each half's clicks and impressions into rates to set beside each other. They explain the result, but conversions decide it, because a title can win clicks for searches that do not buy.

Three guardrails protect the reading:

  • Impressions and conversions of the control products against their own pre-period. This is the spillover check. If the control half falls while the treated half rises, some of the gain is spend moving inside the campaign rather than new demand.
  • Disapprovals and diagnostics warnings in Merchant Center. A rewritten title can break a policy, for example by adding promotional or capitalised text, which the feeds guide lists as a problem in titles. A disapproved product stops serving and spoils its half's reading.
  • Search terms that no longer describe the product. Read the search terms or search category insights for the treated products. A pattern that front-loads a broad word can attract searches the product does not answer.

Design

Randomise at product level, within one category. Use the item group, with all its variants, as the unit, so that sizes and colours of one product never sit in different halves. List every product in the category with steady sales over a recent period, sort them into bands by past sales, and randomise within each band, so both halves carry a similar mix of strong and weak sellers. Run a pre-period with no change and confirm that the two halves move together. Then upload a supplemental feed, keyed on id, that overrides the title for the treated half only.

Design elementChoiceReason
Unit of randomisationProduct item group, with its variants, within one categorySearchers cannot be randomised; products can, and variants must stay together
HalvesTreated with the new title pattern; control on current titlesIsolates the title
BalanceRandom assignment within bands of past salesStops a few best sellers deciding the result
How the change is madeSupplemental feed overriding titles for the treated halfChanges the title without touching the site or the primary feed
Held constantBids, budgets, prices, images, stock and campaign structure in both halvesOnly the title differs
Primary metricConversions per product, each half against its own pre-periodRemoves shared shocks and starting differences

The feeds guide documents the order in which Merchant Center applies feed layers: the primary feed first, then supplemental feeds by id, then rules. Check before launch that no title rule overwrites the supplemental value, or the treated half will quietly keep its old titles.

Both halves stay in the same campaign and the same listing groups, because moving the treated products into a new campaign would change their budget and bidding as well as their titles. That choice is what makes spillover the main design risk. In a Performance Max campaign, or a Standard Shopping campaign on automated bidding, the system can move spend towards the products that start winning, which inflates the treated half and depresses the control. This is general practice rather than documented behaviour, and the control guardrail is how the test measures it. The Performance Max structure guide explains how listing groups and product reports are read, and the per-product figures for both halves come from those reports.

Sample size reasoning

Power comes from the number of products and how steady their conversions were before the test, not from the number of clicks. A category with many products that each convert steadily gives a clear reading quickly; a small category, or one whose products convert in bursts, needs a longer window or cannot answer the question. A few best sellers usually carry most of the volume, and if they land unevenly they swamp everything else, so balance the halves on past sales or leave the best sellers out and test them separately.

The incrementality testing guide sets out the group logic: decide the smallest lift that would change the decision, then work back from the variation between groups to whether the groups available can detect it. The measuring SEO changes guide's control-group method shows the same check in practice: if the two halves did not move together in the pre-period, a difference after the change cannot be trusted. If the category is too small, combine closely related categories that share one title pattern, or accept that the test can detect only a large effect.

Duration

Start with a pre-period long enough to show the two halves moving together. Then upload the supplemental feed and wait until Merchant Center has processed it and the new titles are serving, checking a sample of treated products, before the post-period begins. Run the post-period for whole weekly cycles. Avoid promotions, price changes and stock gaps that affect one half and not the other; if one cannot be avoided, record it and exclude the affected products from both periods.

Reading the results and the decision rule

Read the test once, at the end of the post-period. Check first that no treated title drew a disapproval; fix any that did, and exclude those products from the reading. Then compare conversions per product, each half against its own pre-period, with the interval around the difference. Check the control half's own trend, and only then read the search terms. Write the decision rule into the plan in advance:

ResultDecision
Treated products gain more against their pre-period than the control, and the control has not fallen by the amount gainedRoll the pattern out to the category
The halves do not differKeep the old titles
Treated products gain, but the control falls by about as muchTreat the gain as spend moving inside the campaign, keep the old titles and rethink the pattern
Some treated titles drew disapprovalsFix those titles before reading the test

A winning pattern is a result for this category. The next category may search differently, so retest the pattern there before assuming it carries over.

Pitfalls

Budget moving from control to treated products. The treated half can look better simply because the campaign spends more on it. Read the control against its own pre-period every time, and do not change budgets or targets while the test runs.

Best sellers dominating one half. Balancing on past sales prevents most of this. If one product carries a large share of the category, leave it out and test it on its own.

Promotions or stock gaps in one half. A sale price on a treated product or an out-of-stock control product moves conversions for reasons unrelated to the title. The Shopping and Performance Max playbook gives a named feed owner the product lifecycle decisions; ask that owner to hold promotions and range changes for the category until the reading is done.

Reading before the feed is processed. The post-period starts when the new titles are serving, not when the supplemental feed was uploaded.

The SEO title tag test uses the same page-group logic for organic titles, where pages do not share a budget, and the Performance Max uplift test asks the earlier question of whether the campaign type adds conversions at all.

Example: an invented lighting shop retitles half a category (illustrative)

Illustrative example — not a real result

Ganvoleth Lighting tests a new title pattern on its ceiling lights

Ganvoleth Lighting, an invented online lighting shop, sells 184 ceiling lights through one Performance Max campaign. Their feed titles were copied from page titles such as "Our Brightest Ceiling Light Yet". It plans a pattern of brand, type, fitting and size, such as "Brand Pendant Light – E27 – 40cm". Four best sellers are left out for a later test, and the remaining 180 are banded by past sales and split at random, 90 to each half. A six-week pre-period shows the halves moving together. The supplemental feed is uploaded, the new titles are confirmed serving five days later, and the post-period runs six weeks.

HalfConversions per product, pre-periodConversions per product, post-periodChange
Retitled11.815.0+27%
Control12.111.6−4%

The retitled half gains 3.2 conversions per product, about 290 in total, while the control loses 0.5 per product, about 45 in total, so most of the gain is not spend moved from the control. Click-through on the retitled products rises from 0.8% to 1.1%. Two titles were flagged for capitalised text in the first week and fixed before the reading. Ganvoleth rolls the pattern out to the rest of its ceiling lights and plans a separate test for the four best sellers.

Common mistakes

Feed title tests usually fail on the comparison rather than the titles. The experiment library lists the other designs, including the page-group tests beside this one.

  • Reading the treated half alone. Without the control's trend, a seasonal rise looks like a title effect.
  • Ignoring the control's fall. A gain matched by a loss in the untouched half is spend moving, not demand won.
  • Splitting variants of one product. Sizes and colours of one item group belong in the same half.
  • Letting a feed rule overwrite the new titles. Check the order of feed layers before launch.
  • Changing bids or budgets mid-test. The test then measures the campaign change as well as the titles.
  • Rolling a pattern out to every category on one result. Each category's searches differ; test before extending.

Frequently asked questions

Why split products rather than run the new titles on every product and compare before and after?

Because a before-and-after reading credits the titles with everything else that moved in those weeks: the season, competitors' prices, a change in the campaign's bidding, a promotion. Keeping a random half of the category on its old titles gives a comparison that lives through the same weeks, so shared changes show up in both halves and the difference between them belongs to the titles.

Can the new titles hurt the products that were not changed?

They can, and the design is built to catch it. Products in one campaign share its budget and bidding, so if the retitled half starts winning more auctions, spend can shift towards it and away from the untouched half. That would make the treated products look better and the control products look worse without the category earning any more. Reading the control half against its own pre-period, as a guardrail, shows whether the gain came from new demand or from moving spend.

Should the test cover the best-selling products?

Only if they can be balanced between the halves. A few best sellers usually carry most of a category's sales, and if they land in one half they decide the result on their own. Either split them evenly when the halves are formed, or leave them out of this test and retitle them in a separate, smaller test once the pattern has proved itself on the rest of the range.

Terms used on this page

Tools to try

Frameworks

Also related

Sources

  1. Product data specification — Google Merchant Center Help
  2. Google Ads Help — Google Ads Help

Related