Skip to content
Faisal Mahmud TusherDigital marketing, SEO & analytics
with Faisal Tusher

Customer Experience, Retention & Loyalty › Troubleshooting

Relationship Survey Score Falling

Why a relationship score keeps falling: noise read as change, a sample that shifted, surveys at the wrong moment, rising effort and service failures.
Troubleshootingintermediate17 min readUpdated

First, draw the score's own range and compare who answered with who was asked

The voice of the customer guide explains how the relationship score is built: respondents are sorted into three groups by where their answer sits on the scale, and the net figure is the top group's share less the bottom group's. Because the net figure discards most of the distribution, it jumps about when few people answer, and the guide's first reading rule follows from that: watch the trend and the shape of the answers rather than any one wave, since a move that stays inside the score's ordinary noise tells the business nothing. The way to know the normal range is to measure it. A control chart draws a centre line and limits from the score's own history, with run rules agreed in advance, so a wave counts as a signal only when it breaks the limits or forms a run the rules name. Draw it from every wave the business has, annotate the known changes, and note that a score from a few dozen responses swings widely for purely arithmetic reasons, so the limits should widen when the sample size is small or the waves should be pooled until the base is large enough to read. The brand health report tutorial applies the same discipline to brand measures, reading movement against noise before calling anything a trend.

If the chart flags a signal, the control chart framework's first instruction applies: check the data before the market. For a survey, the data is the sample. The guide is blunt that a survey anyone may answer describes the people who like answering surveys rather than the customers, and its remedy is to set the respondents against the whole base by tenure, value and segment, then weight the groups or report them apart wherever the two disagree. Put the share of the frame and the share of respondents side by side for each group in every wave of the fall. A score that fell because new or low-value customers now make up more of the replies, or because one team began surveying its customers every month and so dropped the contact rule, has not measured a change in how anyone feels.

Only after both checks is it time for the distribution and the comment box. Look at whether the share of top scores fell or the share of bottom scores rose, which are different problems; split by tenure, product, channel and value before concluding, because an overall score can hold while one group grows unhappier; and code the open text so the themes most common among the lowest scorers can be counted against the base. The measuring customer experience hub sets perception beside behaviour and operations for exactly this reason: a relationship score that falls signals a problem without naming it, and the name is usually found in support contacts, repeat contacts and what the same customers do next. This page sits beside the voice of the customer guide, which explains what each score measures, and the tutorial on testing whether survey scores predict churn, which checks a score against behaviour; it starts from a score that has already fallen for several waves and works through noise, sample, timing, effort, service and prediction in that order.

Diagnosis table

Diagnosis

SymptomCheckLikely causeFix
The score moves every wave, the sample is small and the fall is the size of earlier rises nobody celebratedA control chart of the score's full history with limits and run rules agreed before readingA movement within the score's normal range of noise read as a real changeReport the chart rather than the number, pool or widen limits for small waves, and act only on a signal
The fall coincides with more replies from new, low-value or one segment's customers, or with one team sending its own surveyRespondents against the frame on tenure, value and segment, wave by wave, and which teams surveyed whomThe respondents shifted: more new or low-value customers answering, or a contact rule droppedRestore one contact rule across teams, draw the sample on purpose, and weight or report groups separately
The low wave landed just after a price rise, during an outage or before customers had the outcomeThe send dates against the business's own calendar of price changes, incidents and delivery or resolution timesSurveys sent at a distorting moment: after a price rise, during an outage or before the outcome was knownMove the send to a neutral moment, delay transactional surveys until the outcome is known, and annotate the wave
Open text from low scorers names repeating details, being passed around, switching channels or waitingContacts by reason, repeat contacts and transfers in the business's own records, read against the coded themesEffort rising somewhere in the journey: repeated information, handoffs, waitingRemove the step, do it for the customer, tell them first or join the history up, ranked by impact and effort
Detractors cluster among customers with an open or repeated contact, a complaint or a failed deliverySatisfaction by contact reason, repeat contacts within the window and complaints by cause for the wave's respondentsService failures and repeat contacts feeding the detractor end of the scaleRead speed beside repeats, fix the reasons that come back, and run the recovery sequence for every failure
The score fell but churn, repeat purchase and complaints did not move, or the score was never joined to behaviourWhether the score has ever been linked to what the same customers did afterwards, within comparable cohortsA score that was never shown to predict behaviour, so the fall may say little about retentionRun the prediction check with the window fixed in advance and decide what the score is for

Fix, verify and prevent each cause

Noise read as a real change

The voice of the customer guide warns against reacting to small month-to-month movements on small samples, and the control chart framework explains why reviews do it anyway: most reporting treats every movement as news, and teams spend meetings explaining changes that would have reversed themselves. A relationship score is unusually prone to this because the net calculation discards the distribution and because many waves rest on small samples.

Fix: build the chart from the score's own history, with a baseline of ordinary waves, limits set from the routine wave-to-wave movement, and a short set of run rules fixed in advance; use limits that widen when a wave is small, or aggregate waves until the base is large enough; and present the chart in the review in place of the single number.

Verify: the recent falls sit either inside the limits, in which case the review records that nothing has yet been shown to change, or outside them, in which case the remaining rows of the table apply.

Prevent: keep the chart as the standing view of the score, recompute the limits as history builds, restate or restart the baseline whenever the instrument changes, and annotate every wave with the launches, price moves and incidents around it so a later signal can be matched to a cause.

The respondents shifted or a contact rule was dropped

The tutorial on drawing a survey sample from a customer list sets the rule this cause breaks: a survey describes the people who were invited and chose to answer, and nobody else, so the draw decides what the result can claim. Engaged, satisfied customers answer more, which means any change in who is invited moves the score without moving anyone's opinion. One version is a contact rule quietly dropped, when one team begins surveying its own customers on its own schedule and those customers come to dominate the replies, or a frame that filled with new customers after a growth period.

Fix: compare respondents with the frame on the fields the tutorial names, recency, order count, spend, tenure, product and region, for each wave of the fall; where respondents are skewed, weight on the few variables that matter and report weighted and unweighted figures together, and where a group answered too thinly to read, say so rather than weighting it heavily. Then restore the contact rule across every team that sends surveys, so nobody is surveyed twice within a set interval, whichever team is asking, and draw the next wave on purpose, stratified by tenure or value with the draw recorded so the wave after can repeat it.

Verify: the next wave's respondents match the frame within the tolerance the team set, and the score read on the restated, comparable sample sits where the chart expects.

Prevent: record every draw with the brief rather than the extract, add each respondent's survey date to the contact history, and treat a wave drawn differently as a different measurement that is annotated on the chart rather than compared silently.

Surveys sent at a distorting moment

When a survey is sent matters as much as what it asks. The voice of the customer guide's timing rule is to survey after the outcome, not before it, and to keep relationship waves away from price rises and outages unless the reaction to those is the thing being measured: a question about a delivery asked while the parcel is still on its way records what the customer expects, and a question about a support case asked before the fix records only how the first answer felt. A wave that happens to land in one of those moments falls for reasons the next wave will not share.

Fix: set the send dates of the falling waves against the business's own calendar of price changes, incidents, platform changes and delivery or resolution times; move the relationship survey to a neutral point in the cycle; delay each transactional survey until the outcome it asks about has happened; and annotate the distorted wave on the chart rather than deleting it.

Verify: the next wave, sent at the neutral moment, returns inside the score's normal range, and the transactional scores stop tracking the first reply.

Prevent: keep a survey calendar beside the marketing and operations calendars, give the survey owner a line in the incident process so a wave is paused during an outage, and keep relationship and transactional surveys apart, since a relationship question tacked onto each support chat ends up scoring the chat.

Effort rising somewhere in the journey

When the fall is real and the open text names repeating details, being passed between teams, switching channels, long forms or chasing updates, the cause is effort, and the guide to reducing customer effort treats it as a cost the business has moved onto the customer rather than a survey score. Its method is to find effort in the records the business already holds: contacts by reason, where many reasons are effort in disguise; repeat contacts, the clearest evidence that the first contact did not end the customer's work; and the journey and the systems behind it, where history fails to travel across a handoff.

Fix: take the coded themes from the lowest scorers to those records, confirm which kind of effort each is, and apply one of the guide's four fixes: remove the step, do it for the customer, tell them before they have to ask, or join the history up so one record follows the customer across teams and channels; rank the candidates by how many customers they affect and what the fix costs, and start with the ones that affect many and cost little.

Verify: count the evidence that showed the effort before and after, contacts for that reason, repeats about it and the open-text mentions, over comparable periods, and then read the score; fewer contacts about a step is good evidence the step got easier, and the guide makes no claim that lower effort predicts loyalty in general.

Prevent: keep one or two of those counts as a service KPI with an owner, read the open text of every wave against the contact reasons, and walk the journey whenever a system, a policy or a handoff changes.

Service failures and repeat contacts feeding the detractor end

The guide to customer service metrics explains how a service operation can look healthy while customers grow unhappier: a fast first reply that does not solve the problem starts the clock again, and repeat contacts are the clearest sign of a channel answering quickly but badly. Read alone, each metric can be moved without any customer being better served; read by contact reason, a reason with high repeat contacts points to an answer or a process that does not work. When detractors cluster among customers with an open or repeated contact, a complaint or a failed delivery, the score is reporting the operation.

Fix: read satisfaction or effort by contact reason with the response counts beside it, pair every speed metric with the repeat contacts that would expose it, and take the reasons with rising repeats to the people who own the answer or the process; for the failures themselves, run the recovery sequence the diagnosis for customer complaints that are increasing works through, so each failure is acknowledged, fixed and followed up rather than closed.

Verify: repeat contacts for the affected reasons fall within the window the business set, and the detractor share among customers with a recent contact moves back towards the rest of the base.

Prevent: keep contact rate per customer, first-contact resolution and complaint volume by cause on the retention dashboard as the churn metrics guide asks, read them by customer group so they act as a leading indicator, and end the weekly service report with decisions and owners rather than a row of averages.

A score that was never shown to predict behaviour

The voice of the customer guide ends its reading rules with behaviour: link each answer to what that customer went on to do and see whether the low scorers are the ones who leave, because a perception measure that fails that test needs rethinking. The tutorial on testing whether survey scores predict churn runs that check and claims no general relationship between any score and churn; the only answer that matters is the one in the business's own data. A score that has never been joined to behaviour can fall without telling the business anything about retention, and a review that treats the fall as a churn warning may be chasing a measure nobody needs.

Fix: write the question and the decision it serves, confirm the answers can honestly be linked to customers, which the tutorial grounds in ESOMAR's principles of voluntary participation and care with respondents' data as the site's guides state them, fix the outcome and the window before looking, group the scores into a few bands, compare the churn rate across bands within comparable cohorts, add a row for the customers who were invited and did not answer, read the differences with ranges, and repeat on a later period.

Verify: the tutorial's decision table gives the answer: a low band that churns clearly more within cohorts in both periods has earned a place on the retention tree; a gap that appears only in the total, or only once, has not.

Prevent: record the result with its definitions and set a date to repeat the check, because the relationship changes when the product, the customers or the survey change, and keep the score off the headline retention dashboard until it has passed.

Prevention: a control chart, a contact rule across teams and a yearly check of the score against behaviour

Three habits keep the next fall from being misread. The first is the chart. Every relationship and transactional score the business reports is drawn as a control chart with its baseline, limits, run rules and annotations, recomputed as waves accumulate and restarted whenever the instrument changes. Reviews receive the chart, not the number, and the agreed rules decide whether a wave is a signal before anyone offers an explanation. When a rule fires, the order is fixed: the data feed and the sample first, the calendar of price moves, incidents and launches second, and only then the open text and the records.

The second habit is one contact rule across every team, with the draw recorded. The voice of the customer guide sets the rule and the sampling tutorial puts it into practice: once a customer has been invited, no team invites them again until the agreed interval has passed, the survey date is added to the contact history, and each wave is drawn from a written population sentence with strata chosen before the draw and a record that lets the next wave repeat it. Relationship and transactional surveys stay separate instruments with separate charts. Keep the questionnaire fixed between waves; the survey and questionnaire design guide owns wording, scales and order, and a change there is a new series with its own baseline. Nielsen Norman Group's guidance on survey design, as the voice of the customer guide cites it, makes the related point that every extra question costs completion, so the instrument asks only what the business will use. The customer survey wave SOP runs these habits for each wave, from repeating the recorded draw to reading the chart by the agreed rules and storing the record.

The third habit is the open text and the yearly check. Code the comment box of every wave with the codeframe the tutorial on coding open-text answers builds, with definitions, exclusions and examples for each code, a second coder on a portion, and respondents counted per theme against the base, so the themes most common among low scorers can be set against contact reasons and repeat contacts in the same review. Then, once a year or whenever the product or the survey changes, join the wave to what the same customers did next and run the prediction check again, so the business knows whether the score is a leading indicator of leaving or a diagnostic for one process. A score that has passed sits on the retention tree beside the behavioural measures; the diagnosis for a customer churn rate that is increasing reads that behaviour from the other direction, and the two pages together with the customer experience troubleshooting hub cover what customers say, what they do and what the operation did to them.

A removals company finds the fall was mostly a sample change (illustrative)

Illustrative example — not a real result

Hespelthwaite Removals

Hespelthwaite Removals, an invented removals and storage company with about 3,200 moves and 900 storage customers a year, watches its quarterly relationship score fall for three waves, from +34 to +27 to +19. The operations director draws a control chart from the 11 waves the company has: the centre line sits at +31, and the limits, set from the routine wave-to-wave movement on samples of 180 to 260 replies, run from +21 to +41. The first two of those waves sit inside the limits; the third, at +19, sits outside.

Comparing respondents with the frame explains most of it. Storage customers are 22% of the customer base but 51% of the third wave's replies, up from 19% in the first, because the storage team had started emailing every storage customer a survey each month, outside the company's contact rule, and those customers were then also included in the quarterly draw. Reweighted to the frame, the third wave reads +25: still a fall, but a smaller one.

The open text from the reweighted low scorers, coded with the company's codeframe, has one theme in 38% of detractor comments: customers re-sending their inventory list because it was taken at the pre-move survey visit and never reached the removal crew. Repeat contacts tagged to inventory questions have risen from 14 to 41 a quarter over the same period. The company restores the contact rule across both teams, separates the storage survey onto its own chart, and changes the booking system so the surveyor's inventory is attached to the job sheet the crew carries. It also joins each wave to retention: among the previous year's storage respondents, 41% of the lowest band had closed their unit within 12 months against 23% of the highest, so the score keeps its place on the retention tree. The next wave, drawn to the frame, reads +30 on 240 replies.

Common mistakes

  • Reading the number without its range. A fall inside the score's normal variation is not a finding; draw the chart before the review.
  • Treating respondents as customers. The people who answered are the ones who like answering surveys; set them against the frame every wave.
  • Letting each team survey on its own schedule. One dropped contact rule shifts who answers and moves the score with no change in opinion.
  • Surveying before the outcome is known. A survey sent while the parcel is in transit or the case is still open records expectation and the first answer, not the outcome.
  • Changing the question to fix the score. A new instrument restarts the series at the moment the history is needed most.
  • Skipping the comment box. Themes counted against the base and set beside contact reasons are where the cause lives.
  • Assuming the score warns of churn. Until it has been joined to behaviour in the business's own data, a falling score is a question, not a retention alarm.

Frequently asked questions

The score has fallen in three consecutive waves. Surely that is a trend and not noise?

Not necessarily, and the only way to know is to draw the score's own range. A control chart built from the score's history puts a centre line and limits around what it normally does, with rules agreed in advance for what counts as a signal: a point outside the limits, a run of points on one side of the centre line, or a steady climb or fall over a length the team fixed beforehand. Three falls can sit entirely inside the limits when the sample is small, because the score moves erratically on small samples, and three falls can also be a run that the rules flag. Draw the chart before the review, then check the data feed and the sample before anything else, because a change in who answered looks exactly like a change in how customers feel.

Should we change the survey question or the scale to stop the fall?

No. A change to the question, the scale, the invitation or the timing creates a new measurement that cannot be compared with the old one, so the series would restart at the moment the business most needs its history. Keep the instrument fixed while the cause is found, and record any change to it afterwards with the date, so the chart's baseline can be restated or restarted from that point. Wording and scale faults are real and belong to the survey design guide, but they explain a score that was always odd rather than one that recently fell.

If the fall is real, what should we do about it first?

Find which customers and which part of the journey it belongs to before deciding anything. Split the score by tenure, product, channel and value, read the coded open text for the themes most common among the lowest scorers, and set those themes against the business's own records of repeat contacts, transfers, waits and failed deliveries. A fall that belongs to one segment and one theme has an owner and a fix; a fall spread evenly across everyone usually points to a moment that distorted the survey or to a change in who answered. Then close the loop with the customers who scored low and the team that owns the theme, and join the wave to what those customers do next, so the business learns whether the score was warning it of anything.

Terms used on this page

Frameworks

Where this page is used

Sources

  1. ESOMAR — ESOMAR
  2. Nielsen Norman Group articles — Nielsen Norman Group

Related