I read something interesting yesterday which made a lot of sense.
The Five Star rating system is ubiquitous across many fields; from restaurants, to hotels, to Uber rides, to college athletes. What is interesting is that even though the scoring system is always the same, it is applied very differently.
Take Uber rides, which has a Pareto-like distribution. When I worked for Uber (many years ago), drivers had to maintain a 4.7 rating to stay on the road, which required 92%+ rides to be rated 5 stars. In practice, 5 star just meant “no issues or better”, whereas lower scores indicated bad practices. Meanwhile, only 0.01% of high school football players in the USA are classed as 5 Star Recruits for college. Finally, you have hotels, which use the Star system to indicate price rather than quality. In this instance, you could go to 3 Star hotel and have a better experience than at a 5 Star hotel, even if the latter was more “luxurious”.
When you have vastly different systems at play, you get some issues emerge. When my wife and I are looking at Google Maps for places to eat, a score of sub 4 is almost always a no-go, 4.4 is promising, and 4.8+ looks like a must-do. But that’s just a rule of thumb, it doesn’t always work out.
The improved system I saw proposed was as such:
2 or 3 Star: Just select
4 Star: Select from a drop down menu as to why you liked it (service, quality, price etc)
1 or 5 Star: Write a 160 character message about why you feel this way.
This system not only create a more useful scoring system (it is more likely to resemble a Gaussian distribution), but provides additional richness to the context. Not only does it benefit the prospective consumer, but provides much needed feedback to providers. The downside? No one is going to do it. I think I have review 3 places on Google in my life, and only about 10% of my Uber Rides (usually 5 stars). Ain’t nobody got time to write a tweet about how nicely Josh had cleaned his Camry’s interior.
One final point. If you are looking for a way to get better feedback quickly, ask someone to rate it out of ten, but they can’t pick 7. This removes the lazy answer, and gets people to think. I have used it a number of times in many different situations, and it is a handy little heuristic.
Leave a Reply