Hypothesis
If a random half of the products in a category get feed titles rewritten to the category's search-matched pattern through a supplemental feed, those products will win more relevant impressions and conversions than the untouched half, because the title is the main text matched against searches, unless the campaign simply moves budget away from the untouched products. The decision behind it is concrete: rewrite the product feed titles in a category to a search-matched pattern, or keep the current titles.
The guide to product feeds and Merchant Center sets the title patterns and explains why they matter, stating from Merchant Center's product data specification that the title is the main matching text, that truncation in the ad happens from the end and that promotional text is not allowed in titles. Its own illustrative retitling is read before and after. That reading is the usual one, and it cannot separate the new titles from the season, competitors and the campaign's own changes over the same weeks. This design keeps a random half of the category on its old titles so that the comparison lives through the same weeks.
Write the pattern, the category, the halves and the limits on each guardrail into the experiment hypothesis template before the supplemental feed is uploaded. The experimentation loop puts that step before the build, which matters here because a feed change is quick to make and tempting to roll out on a hunch.
Metrics and guardrails
The primary metric is conversions per product, or conversion value per product where products differ widely in price, in the treated half against the control half, each relative to its own pre-period. Take each half's conversions per product in the post-period, divide by the same half's figure in the pre-period, and compare the two ratios. Reading each half against its own past removes the starting difference between them; reading the halves against each other removes anything that moved the whole category. The measuring SEO changes guide sets out the same logic for pages, and it transfers to products unchanged.
Report impressionsOne display of an ad or search listing, as counted by the platform., CTRShare of impressions that result in a click. and CVRShare of visits or clicks that convert. per half as diagnostics. A title that matches more relevant searches should show more impressions on specific queries and a higher click-through on them; the CTR calculator turns each half's clicks and impressions into rates to set beside each other. They explain the result, but conversions decide it, because a title can win clicks for searches that do not buy.
Three guardrails protect the reading:
- Impressions and conversions of the control products against their own pre-period. This is the spillover check. If the control half falls while the treated half rises, some of the gain is spend moving inside the campaign rather than new demand.
- Disapprovals and diagnostics warnings in Merchant Center. A rewritten title can break a policy, for example by adding promotional or capitalised text, which the feeds guide lists as a problem in titles. A disapproved product stops serving and spoils its half's reading.
- Search terms that no longer describe the product. Read the search terms or search category insights for the treated products. A pattern that front-loads a broad word can attract searches the product does not answer.
Design
Randomise at product level, within one category. Use the item group, with all its variants, as the unit, so that
sizes and colours of one product never sit in different halves. List every product in the category with steady
sales over a recent period, sort them into bands by past sales, and randomise within each band, so both halves carry
a similar mix of strong and weak sellers. Run a pre-period with no change and confirm that the two halves move
together. Then upload a supplemental feed, keyed on id, that overrides the title for the treated half only.
| Design element | Choice | Reason |
|---|---|---|
| Unit of randomisation | Product item group, with its variants, within one category | Searchers cannot be randomised; products can, and variants must stay together |
| Halves | Treated with the new title pattern; control on current titles | Isolates the title |
| Balance | Random assignment within bands of past sales | Stops a few best sellers deciding the result |
| How the change is made | Supplemental feed overriding titles for the treated half | Changes the title without touching the site or the primary feed |
| Held constant | Bids, budgets, prices, images, stock and campaign structure in both halves | Only the title differs |
| Primary metric | Conversions per product, each half against its own pre-period | Removes shared shocks and starting differences |
The feeds guide documents the order in which Merchant Center applies feed layers: the primary feed first, then
supplemental feeds by id, then rules. Check before launch that no title rule overwrites the supplemental value, or
the treated half will quietly keep its old titles.
Both halves stay in the same campaign and the same listing groups, because moving the treated products into a new campaign would change their budget and bidding as well as their titles. That choice is what makes spillover the main design risk. In a Performance MaxA Google Ads campaign type that runs one goal across all Google inventory from shared assets. campaign, or a Standard Shopping campaign on automated bidding, the system can move spend towards the products that start winning, which inflates the treated half and depresses the control. This is general practice rather than documented behaviour, and the control guardrail is how the test measures it. The Performance Max structure guide explains how listing groups and product reports are read, and the per-product figures for both halves come from those reports.
Sample size reasoning
Power comes from the number of products and how steady their conversions were before the test, not from the number of clicks. A category with many products that each convert steadily gives a clear reading quickly; a small category, or one whose products convert in bursts, needs a longer window or cannot answer the question. A few best sellers usually carry most of the volume, and if they land unevenly they swamp everything else, so balance the halves on past sales or leave the best sellers out and test them separately.
The incrementality testing guide sets out the group logic: decide the smallest lift that would change the decision, then work back from the variation between groups to whether the groups available can detect it. The measuring SEO changes guide's control-group method shows the same check in practice: if the two halves did not move together in the pre-period, a difference after the change cannot be trusted. If the category is too small, combine closely related categories that share one title pattern, or accept that the test can detect only a large effect.
Duration
Start with a pre-period long enough to show the two halves moving together. Then upload the supplemental feed and wait until Merchant Center has processed it and the new titles are serving, checking a sample of treated products, before the post-period begins. Run the post-period for whole weekly cycles. Avoid promotions, price changes and stock gaps that affect one half and not the other; if one cannot be avoided, record it and exclude the affected products from both periods.
Reading the results and the decision rule
Read the test once, at the end of the post-period. Check first that no treated title drew a disapproval; fix any that did, and exclude those products from the reading. Then compare conversions per product, each half against its own pre-period, with the interval around the difference. Check the control half's own trend, and only then read the search terms. Write the decision rule into the plan in advance:
| Result | Decision |
|---|---|
| Treated products gain more against their pre-period than the control, and the control has not fallen by the amount gained | Roll the pattern out to the category |
| The halves do not differ | Keep the old titles |
| Treated products gain, but the control falls by about as much | Treat the gain as spend moving inside the campaign, keep the old titles and rethink the pattern |
| Some treated titles drew disapprovals | Fix those titles before reading the test |
A winning pattern is a result for this category. The next category may search differently, so retest the pattern there before assuming it carries over.
Pitfalls
Budget moving from control to treated products. The treated half can look better simply because the campaign spends more on it. Read the control against its own pre-period every time, and do not change budgets or targets while the test runs.
Best sellers dominating one half. Balancing on past sales prevents most of this. If one product carries a large share of the category, leave it out and test it on its own.
Promotions or stock gaps in one half. A sale price on a treated product or an out-of-stock control product moves conversions for reasons unrelated to the title. The Shopping and Performance Max playbook gives a named feed owner the product lifecycle decisions; ask that owner to hold promotions and range changes for the category until the reading is done.
Reading before the feed is processed. The post-period starts when the new titles are serving, not when the supplemental feed was uploaded.
The SEO title tag test uses the same page-group logic for organic titles, where pages do not share a budget, and the Performance Max uplift test asks the earlier question of whether the campaign type adds conversions at all.
Example: an invented lighting shop retitles half a category (illustrative)
Ganvoleth Lighting tests a new title pattern on its ceiling lights
Ganvoleth Lighting, an invented online lighting shop, sells 184 ceiling lights through one Performance Max campaign. Their feed titles were copied from page titles such as "Our Brightest Ceiling Light Yet". It plans a pattern of brand, type, fitting and size, such as "Brand Pendant Light – E27 – 40cm". Four best sellers are left out for a later test, and the remaining 180 are banded by past sales and split at random, 90 to each half. A six-week pre-period shows the halves moving together. The supplemental feed is uploaded, the new titles are confirmed serving five days later, and the post-period runs six weeks.
| Half | Conversions per product, pre-period | Conversions per product, post-period | Change |
|---|---|---|---|
| Retitled | 11.8 | 15.0 | +27% |
| Control | 12.1 | 11.6 | −4% |
The retitled half gains 3.2 conversions per product, about 290 in total, while the control loses 0.5 per product, about 45 in total, so most of the gain is not spend moved from the control. Click-through on the retitled products rises from 0.8% to 1.1%. Two titles were flagged for capitalised text in the first week and fixed before the reading. Ganvoleth rolls the pattern out to the rest of its ceiling lights and plans a separate test for the four best sellers.
Common mistakes
Feed title tests usually fail on the comparison rather than the titles. The experiment library lists the other designs, including the page-group tests beside this one.
- Reading the treated half alone. Without the control's trend, a seasonal rise looks like a title effect.
- Ignoring the control's fall. A gain matched by a loss in the untouched half is spend moving, not demand won.
- Splitting variants of one product. Sizes and colours of one item group belong in the same half.
- Letting a feed rule overwrite the new titles. Check the order of feed layers before launch.
- Changing bids or budgets mid-test. The test then measures the campaign change as well as the titles.
- Rolling a pattern out to every category on one result. Each category's searches differ; test before extending.
