Five stars

The five star rating system is commonly used to rate many things such as hotels, films, music, and players on Football Manager games.  Football Manager got me thinking about the five star system when I discovered that on the 2011 version of the game, the best player in the game, Lionel Messi, was 4.5 stars.  How can the very best of thousands of players still not be in the equivalent of the 90-100% range?!

There are two problems with the five star system as a ranking system – one obvious and practical, one less obvious and more theoretical.

Firstly, things are often given half stars, such as 3.5 stars out of 5.  Why not just rate things out of ten instead, and make everyone’s life easier?

Secondly, upon examination of the five star system, I see that there are several possible ways to use it, which make various amounts of sense.

The first way to use this system is with a uniform distribution – so the top 20% of films get 5 stars, the next 20% get 4 stars, and so on.  This is the most helpful way, because you can immediately tell roughly how good a film is from the number of stars it has.  Sadly, this is also the least used way.  It is also impractical to use for something like films – because how can one tell how a film will compare to as yet unreleased films?

The second way is with a normal distribution – so the many films get 3 stars, some get 2 and 4, and few get 1 and 5.  This is less useful, because so many films get the standard 3 stars, but more often used than the first way.  It also suffers from the same drawback as the first way.

The third way is with an arbitrary distribution – so the reviewer or critic basically guesses the star number based on how good they thought the film was.  This is unhelpful because there is no standardisation – one critic may give a ‘reasonable’ film 3 stars, another may give a ‘reasonable’ film 2 stars.

The fourth way is the checklist system, often used for hotels – so a hotel gets four stars if it has certain facilities or services.  This is more helpful, but can’t be used for things like films or music.

The five star system is generally very unhelpful, because you never know the criteria the reviewer is using (unless you have read all the reviews by a reviewer, which is just silly).  Now I think about it, the same could be said for out-of-ten or percentage rating systems.  The numbers are fairly meaningless.  Therefore giving a film a score, out of anything, is a waste of time, unless (a) you are going to give a score to all the films you have seen, and (b) people are going to look at all those scores – then a ranking list could be produced by the reader to helpfully indicated what you think of films compared to each other.  In which case, forget the scores and just produce a ranking list!  (Some readers will be smiling at this point as I attempt to justify all my ranking lists).

A much more helpful way of communicating thoughts on films etc is with a full review.  If people would prefer a score out of 5, or 10, or 100 because they are too lazy to read a review, tough.  An isolated score is largely unhelpful.  If you want my score out of 10 for a given film, you will have to listen to my score for every film I have ever watched.  Good luck with that.