Living and Learning

I Give Star Rating Systems Two Stars

When we give a star rating to a business or a play or a product, often that is all we do. Much of the time, we don’t analyze why we gave that rating or justify it. If we do give an explanation on sites such as Angie’s List or HomeStars it is often because we were either very pleased or very displeased. There are not a lot of explanations in the mid-range because those are the hardest to put into words. It takes time and effort to turn a sense of nuanced satisfaction or disgruntlement into a detailed analysis, so most of us don’t bother.

Star Rating SystemI have reached the point where I no longer feel inclined to provide feedback on services for a few reasons. First, I have had one experience when a service provider phoned to yell at me about my feedback, and another when the provider called within hours asking for an explanation. In both instances, I felt intimidated.

Second, the rating systems can make or break small businesses, and that is simply unfair. Of all the clients they have, only a small percentage will submit feedback. If a significant proportion of the people who provide feedback do so only because they are unhappy, then the star system provides a skewed result. Those results on review sites then become the public face of the business.

Third, star ratings can be statistically invalid. I don’t know much about statistics, but I do know that it takes more than a few data points to make some sort of projection. Pollsters get feedback from thousands of people before they are able to make a determination on something, and even then, it has to be stated with qualifiers about degrees of probability. Feedback on businesses and cultural products do none of those things. They are simply the opinions of one or a few people, but when they are placed in public media they can be perceived to be just as valid as exhaustive research.

Rating SystemFourth, individual assessments of the work of a person or a business on public websites give the reviewer a megaphone. If they were to talk directly to the subject of their feedback, they would be able to engage in debate and correction if it is called for. The perception could be clarified, the work could be re-evaluated or redone, and the praise could be enjoyed in person. Very little, if any, of that interaction happens on feedback websites. What could be private becomes public and has far more influence than it deserves.

Teachers and instructors have to grade students’ work all the time, and many find the task challenging. Not only is it time-consuming to read through students’ work or to develop tests, but it is also troubling to have to reduce all the work to a single letter or number. It means so much to the student but on its own, the score looks inadequate. And, it is.

Grading GIF

Grading from tenor.com

The score doesn’t tell you how much work went into achieving it or how much the student has improved over time. It doesn’t tell you how much risk was taken in coming up with the finished work or how difficult it was for the student to find the time to do it. It only tells you what all the assignments, all the learning, all the reading, all the writing, and all the discussions boil down to. It is the concentrate, not the juice.

When possible, educators give their students an explanation to go with the scores. They show where the work falls short of expectations, and where there were wrong answers or faulty logic. Many will also show where the work was well done and why. Ultimately, they want the student to succeed and they try to point them in that direction and to encourage them.

Star rating systems on small businesses and local arts do none of these things. A star rating might have some value when thousands of responses are collated, such as on Netflix, but when tied to a single person or local business a few low scores can be devastating. Not only do they demoralize the people targeted, but they fail to guide them to greater success. If I were grading them, star rating systems would not pass.

10 replies »

  1. I stand under correction but insurance companies require the rating as part of the paperwork that service providers hand in.Out here in SA at any rate.
    All such contracted work has to be BEE compliant ( Black Economic Empowerment) and it seems. that as long as this criteria is fulfilled no other checks or balances seem to be in place.
    Sometimes one can hit the jackpot but in general from my own experience It is usually frustrating, involving follow up calls to have the job re done or more work to fix up shortfalls.
    And for this reason I do not enjoy ticking those boxes at all.

  2. The medical industry is rewarded with high starr ratings by Medicare for reimbursement. That is why I fill them out after each MD visit. You can go online and see what clinics and Hospitals , Home Health agencies, Dental office have high ratings for care and outcomes to choose what fits you.

    • It is a very useful system when there are thousands of responses. The problem I ran into was with local businesses that get just a few responses. With enough numbers, the statistics make the star rating fairly representative. With just a few, they don’t

  3. I have never had anyone contact me for a poor rating, but then I only answer my phone if it is a known contact and of course any one from Canada! I agree the smaller business’s rely on it more. I do find myself looking at reviews before I decide what company to use for my home.

Please leave a comment.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.