Em-vestigation: Into the Review Score

em-vestigation-into-the-review-score

We’ve all done it. Each and every one of us at one point or another has clicked on a review and scrolled all the way to the bottom, satisfied by looking only at the score, grade, number of stars, or whatever scale your publication of choice may choose. Maybe we didn’t have the time to read the full review when it crossed our feed or maybe the game or movie didn’t pique enough interest in the first place to warrant that much attention. Whatever the reason it doesn’t make us bad people. The internet age has spoiled us with all the information we could ever want at our fingertips, causing us to demand our news in 140 characters or less and our reviews summed up by some point system.

A quick search of almost any major outlet that publishes video game reviews will reveal that everything from Triple-A juggernauts, to Indie heartthrobs, to throw-away Mobile titles are rated against the same parameters. How do we remedy the idea that Destiny, The Unfinished Swan, and Flappy Bird are even mentioned together in a critical space? Do we create a different scale for each platform? Does it make more sense to group reviews by genre? I don’t have the answers but I do know that many sites have already taken a progressive step towards creating what I hope will become a scoreless future.

Flappy Bird

My first exposure to review scores came in the pages of GamePro Magazine. Their method was to rate games from 1 to 5 on a series of different attributes including Controls and Visuals. Even back then I was confused when games on different platforms would achieve a perfect score in the “graphics” category. How did a Gameboy game and a Super Nintendo game both score a 5? There was no clear indication that the scale would shift depending on the platform, but to me it inferred the game was being compared to the best visuals offered on its respective console. This issue would be even more complex today considering the varying art styles and polished 8 or 16-bit graphics in many modern Indies.

… I’m not the right person to review this game …
Throughout my experience with PS Nation I’ve changed my ideals about the review process several times as I’ve become more adept at critiquing. Before I wrote my own I would have been repulsed by the idea of a rating-less review. I’d be lost at the end of the writer’s conclusion without the reductive, 2-digit cherry on top. I was a big fan of our letter grade system for a long while too until I began to feel that the American school grades we’re so used to in the States not only convoluted the score I was trying to convey but also did not translate well to international readers. The 20-point scale we have now works, but 90% of games fall onto 30% of the scale. I feel as though I’d be hard pressed to find an outlier that isn’t somewhere in between a 6.5 and a 9.5.

Destiny_20140928192711

So many outside factors other than solely what’s on the disc or in the file can affect a reviewer’s approach, be it consciously or subconsciously. We can’t pretend that the state of the industry, the type of game we’re playing, our own particular bias, and a myriad of other factors don’t influence the final decision. If I dislike first-person shooters and I don’t typically play online, how am I to fairly assess the next installment of Call of Duty? To play devil’s advocate, is it completely impartial to have a fan of the series review the game? It’s likely that the reviewer is already in favor of the title based on the franchise’s history. It is also possible that the aspect they’ve always loved about the series takes a turn for the worse, resulting in a lower score even if the opinion is more “informed”.

… let’s focus on and discuss a review’s text …
You’ve all heard Glenn on the podcast saying “I’m not the right person to review this game”. That is the reason why Andy and Jason (God bless them) take on the brunt of the RPG critiquing. They can give a 70+ hour grind-fest the “fair shake” that many of us cannot. Or can they? Some of it depends on the reader as well. If the RPG fans are recommending an RPG to fellow RPG fans, then all is right with the world. But if the game is using a simplistic and accessible leveling system that the hardcore would scoff at, then we have a problem. For whom is that 9.0 suggested? My point is that all of this information is clearly expressed with thousands of words before the score is even revealed, causing the score to not only devalue itself, but the in-depth analysis leading up to it as well.

Dragon Age

I understand that we live a world where scores are still necessary. It may in fact be unfortunately true that Metacritic ratings are taken into consideration by publishers when deciding if a game deserves a sequel. I even find myself speaking in numbers when my opinion of the new hotness is asked of me. For Evolve, “a multiplayer, teamwork focused first-person shooter with human-controlled monsters and hunters” is an infinitely better descriptor than “mid 7’s”.

Reviewers play games before the internet’s popular opinion is projected upon their existence. Without the influence of others who have played the game, many reviews hit the second an embargo lifts, offering up that unique person’s feelings about their particular experience at that point in time with that work of art. We’re forced to reduce that experience into a rating that will inevitably become the focus of an entire dissertation. Until the industry as a whole is ready to kick these scores to the curb, let’s focus on and discuss a review’s text, not its forcefully whittled down, couple-syllable “rating”.

Twitter Digg Delicious Stumbleupon Technorati Facebook
  • John Montegudo

    Totally agree. Review numbers are not accurate.

  • Jahonius

    Excellent insight!
    As much as I like and respect reviewers’ point of view now, I used to bitch and moan about review scores just a few years back when I started reading them.

    The recent The Order : 1886’s divided reception was just the case. It was a clear example where reviewers getting so much shit from the readers no matter the scores given out.

    I don’t regret playing hard-earned cash for games like RE5 and 6 despite the review scores since I definitely enjoyed both. Games like Dead Island, however were intolerable for my taste, again despite the positive review scores.

    I think games nowadays cater to so many different audience that “I’m not the best person to be reviewing this game” is a criteria all editors consider before assigning reviewers, especially for bigger sites.

  • Keith Dunn

    I give this article a 3 cubed!

  • Roger Cannon

    Yes. I’ve read too many reviews that were negative for reasons that don’t matter to me. The proof is in the pudding, the reason someone likes or dislikes a game is in the text, not in the numbers. Like a pie chart in USA Today, it means little to thinking consumers.

  • George Nash

    I don’t really want to read long reviews all of the time. So I do rely on the scores for a lot of my game buying decisions.

    For the most part reading the conclusion and looking at the score is enough for me to decide. I also tend to visit smaller sites that have the same reviewers over and over. Like psnation.com this gives me the ability to get to know the likes and dislikes of individual reviewers. Some times a high score requires a closer look because I know more about the reviewers likes and dislikes. Other times a low score is not given as much attention for the same reason.

    I personally feel even if a review score is an inaccurate judgement. At least its a judgement that can be quantified quickly.