The five star rating system is commonly used to rate many
things such as hotels, films, music, and players on Football Manager games. Football Manager got me thinking about the
five star system when I discovered that on the 2011 version of the game, the
best player in the game, Lionel Messi, was 4.5 stars. How can the very best of thousands of players
still not be in the equivalent of the 90-100% range?!
There are two problems with the five star system as a
ranking system – one obvious and practical, one less obvious and more
theoretical.
Firstly, things are often given half stars, such as 3.5
stars out of 5. Why not just rate things
out of ten instead, and make everyone’s life easier?
Secondly, upon examination of the five star system, I see
that there are several possible ways to use it, which make various amounts of
sense.
The first way to use this system is with a uniform
distribution – so the top 20% of films get 5 stars, the next 20% get 4 stars,
and so on. This is the most helpful way,
because you can immediately tell roughly how good a film is from the number of
stars it has. Sadly, this is also the
least used way. It is also impractical
to use for something like films – because how can one tell how a film will
compare to as yet unreleased films?
The second way is with a normal distribution – so the many
films get 3 stars, some get 2 and 4, and few get 1 and 5. This is less useful, because so many films
get the standard 3 stars, but more often used than the first way. It also suffers from the same drawback as the
first way.
The third way is with an arbitrary distribution – so the reviewer
or critic basically guesses the star number based on how good they thought the
film was. This is unhelpful because
there is no standardisation – one critic may give a ‘reasonable’ film 3 stars,
another may give a ‘reasonable’ film 2 stars.
The fourth way is the checklist system, often used for
hotels – so a hotel gets four stars if it has certain facilities or
services. This is more helpful, but can’t
be used for things like films or music.
The five star system is generally very unhelpful, because
you never know the criteria the reviewer is using (unless you have read all the
reviews by a reviewer, which is just silly).
Now I think about it, the same could be said for out-of-ten or
percentage rating systems. The numbers
are fairly meaningless. Therefore giving
a film a score, out of anything, is a waste of time, unless (a) you are going
to give a score to all the films you have seen, and (b) people are going to
look at all those scores – then a ranking list could be produced by the reader
to helpfully indicated what you think of films compared to each other. In which case, forget the scores and just
produce a ranking list! (Some readers
will be smiling at this point as I attempt to justify all my ranking lists).
A much more helpful way of communicating thoughts on films
etc is with a full review. If people
would prefer a score out of 5, or 10, or 100 because they are too lazy to read
a review, tough. An isolated score is
largely unhelpful. If you want my score
out of 10 for a given film, you will have to listen to my score for every film
I have ever watched. Good luck with
that.