The Rating Only Half Your Customers Can Give
Satisfaction ratings can only be submitted by a requester with an account, through the self-service path. Everybody who reached the desk without one — which is the group that got a tracking link instead — is missing from the average, and the average does not say so.
A satisfaction score is a number about a population, and the number is the part everybody looks at. The population is the part that decides whether the number means anything, and it is almost never printed next to it.
The position, stated first
The average is computed correctly. It is computed over the requesters who hold an account and use the self-service path, because that is the only place a rating can be submitted from. If a large share of your requesters are external, the figure describes a different desk than the one you run — and nothing on the screen tells you which.
One way in
A rating is one to five, with an optional comment, and it can only be given once a ticket is resolved or closed. All of that is sensible.
It is submitted through exactly one path: the signed-in self-service interface, which checks that the person submitting owns the ticket. There is no web route for it at all.
An external requester cannot pass that ownership check, because they do not have an account. That is not an oversight in the rating feature — it is the deliberate design of external requests, which exist precisely so somebody can raise and follow a ticket without registering.
A month on a mixed desk
Illustrative. The arithmetic is right in every row. What is missing is any indication on the screen that seventy per cent of the tickets were never eligible.
A response rate tells you who chose not to answer. It cannot tell you about the people who were never asked.
Why this is worse than a low response rate
Every satisfaction programme has a response-rate problem, and everybody knows it. People who are mildly satisfied do not fill in forms; people at the extremes do. That bias is well understood and analysts correct for it instinctively.
This is a different thing. It is not that some people declined — it is that a whole class of requester was structurally excluded, and the exclusion correlates with something that matters. Requesters without accounts are typically external customers, and external customers are typically the ones whose satisfaction the business is actually being measured on.
So the sample is not merely small. It is drawn from the population you were least worried about.
What the number is still good for
Movement. A figure computed the same way every month over the same population is a legitimate trend even when its level is unrepresentative. If it drops, something changed, and that is worth knowing.
What it is not good for is comparison — against a target, against another desk, or against an industry figure — because all of those assume a population this one does not have.
Almost all requesters have accounts
Use it normally
An internal IT desk is the clean case. The population is the desk, and the number means what it appears to mean.
A mix of internal and external
Use it as a trend only
Never as a level, and never against a target. Publish the count alongside it so nobody forgets.
Mostly external requesters
Do not report it
A figure over twelve per cent of tickets, presented as satisfaction, will be quoted by somebody in a board pack within two months.
You need a real number
Measure outside the system
A follow-up email with a link is a survey, not a feature, and it will cover everybody.
What AWRA OpsHub does today
- A one-to-five rating with an optional comment on a resolved or closed ticket.
- A check that only the ticket's owner can rate it.
- An average and a response count in the module reports, and an average on the support dashboard.
- A reopened count per ticket, incremented properly, and used in reporting and in the risk score.
What it does not do
- Any way for a requester without an account to submit a rating. There is no web route for rating at all.
- Any prompt or reminder asking a requester to rate a resolved ticket.
- Any indication beside the average of how many resolved tickets were eligible to be rated.
- Segmentation of satisfaction by agent, category or channel.
- Any follow-up survey mechanism.
Not ours, by choice
- This was found by auditing our own code for this batch. The rating feature works exactly as designed; what is missing is the eligible population beside the figure, and that omission is ours.
- For an internal desk where every requester has an account, none of this applies and the number is straightforwardly useful.
- Nothing here is Colombian, Brazilian or Chilean. Latin America is here because mixed consumer-and-business desks are common there, and a mixed desk is exactly where an account-gated rating misleads.
Two, and the second one is a label rather than a feature
One of these closes the gap and the other one makes it visible. We would do the label first, because it is honest immediately.
Rating from the tracking link
The external requester already receives a signed link to a page showing their ticket. Adding a rating control to that page, authorised by the same signature, covers everybody who was ever told about their ticket — which is the whole population rather than the account-holding part of it.
The eligible count, printed beside the average
Ratings received, out of resolved tickets, out of resolved tickets that could have been rated. Three numbers instead of one. This is not a feature so much as a refusal to present a figure without its denominator, and it would have made the gap self-evident years ago.
How it works: you describe the requirement, we return a written scope, timeline and cost, and once agreed it is built into your environment and maintained as part of the product.
Talk to us about satisfaction measurementFour questions about any satisfaction figure
Who is able to submit a rating?
A good answer sounds like
Every requester.
What it actually means
The question that found this. "Customers" is not an answer; ask which customers can reach the control.
What is the denominator on that average?
A good answer sounds like
Resolved tickets, stated.
What it actually means
An average with no denominator on the screen is a number waiting to be misquoted.
How many resolved tickets could not be rated?
A good answer sounds like
Zero, or a number they know.
What it actually means
Ours is most of them on a mixed desk, and nothing displays it.
Is anybody prompted to rate?
A good answer sounds like
Yes, on resolution.
What it actually means
Without a prompt the response rate is whoever went looking, which is a strong bias on top of the structural one.
Ask who could have answered
Before you put a satisfaction figure in front of anybody, find out how many of your resolved tickets were eligible to produce one. It is a single query and it changes how the number should be read.
Talk about support measurementFrequently asked questions
Can I ask external customers to create accounts so they can rate?
You can, and most will not. The reason external requests exist as a first-class path is that requiring registration loses you the request, and trading a ticket for a rating is a bad exchange.
Is the reopened count reliable?
Yes — it is incremented on the ticket itself when a ticket is reopened, and it is read by the dashboards and the risk scoring. It is a genuinely useful quality signal and it does not have the population problem, because it does not depend on anybody choosing to respond.
Should we stop showing the average?
Not necessarily — as a trend over a stable population it is legitimate. What we would stop doing is showing it without the count of ratings and the count of resolved tickets beside it, because a bare average invites exactly the comparison it cannot support.
