Current

Calling out the Game Reviewers

Jeff Minter’s Unity Space Giraffe comes out this week. It’s a highly anticipated game in some circles. OXM got its hands on it early and reviewed it, giving it a 2 out of 10. As expected, some people in those circles of anticipation cried foul. That’s fine.

What isn’t fine is having the game’s creator, Jeff Minter himself, out the reviewer as a “rubbish” player.

In short, the leaderboards show that OXM Dan was playing the game in about the most shit way possible, and being absolutely crap at it. The leaderboards also show a lot of other people playing the game rather well, so we know it’s not the game at fault here.

Clearly the reviewer was just “shit” as the game was faultless. There is no criticism to be taken here. The game could never have been counter-intuitive or too hard. It could not have a bad user interface or a lousy tutorial or a steep learning curve. It’s all the fault of a shitty game player and his review is instantly voided because of it. His opinion has no merit. This kind of ego stroking serves no one and makes Minter out as an arrogant prick.

The whole thing reminds me of the Gamespot “Savage” review controversy. Greg Costikyan wrote about that saying:

Personally, it rarely takes me an hour to decide a game sucks. Or rocks. It’s only the ones in between that might take more time for me to judge. Albeit, if I love a game, I’ll play it for a long time, both because I love it, and because I want to figure out why. And that might make for an informed review–but reviews inherently aren’t informed. There isn’t a zine (or website) in the field that publishes intelligent informed game criticism; bare reviews are all we can expect. (A rant for another time, perhaps.)

So… I almost don’t care how long this guy took to form his opinion. He formed his opinion, he wrote it up for whatever pittance Gamespot pays, and that was all that could reasonably be expected of any professional in this field, given what a pittance they are paid.

A review, in the end, is opinion. If a game is too hard for the player, they can complain about that. If a game is boring and dull in the first hour, they shouldn’t be expected to continue playing until “it gets good” around hour twenty. That’s ludicrous. If it takes an hour to get a sense of a game, then it’s valid. And if a review is opinion, then the criticisms that Space Giraffe is Too visually noisy to figure out what going on. and Actually made us angry when we played it. are as valid coming from someone that sucks at the game as they are coming from an expert.

Calling out the reviewer for “sucking” is childish and immature and, worst of all, sets a bad precedent. Would Minter have called out this reviewer for “sucking” if, having played the exact same way, he gave the game a 9 out of 10? I doubt it. It’s a deliberate attempt to undermine a negative review and to discredit the opinion of a reviewer. Nothing more.

Tracking reviewer habits in an online game opens up a whole new can of worms and raises a lot of questions about trust, anonymity and opinion. It’s just more to consider in the quagmire of PR/media, PR/developer and developer/media relations, media exclusives, and all sorts of issues regarding games journalism. I really don’t know what the answers are, especially since I’m an outside observer, but I can say that I’m clearly in favour of less whining by developers when a review doesn’t suit their biased views. It lowers my opinion of the developer.

It’s about as bad as hyping a game for years and then canceling it unceremoniously.

COMMENTS

Brett writes (August 20th, 2007 at 19:08):

Yea, that is disappointing and very uncool. I can’t blame the guy for having his feelings hurt, but it doesn’t mean that basic adult professionalism has to go by the wayside. More interesting to me is that the social features of the Xbox 360 apparently really do allow game makers to monitor and possibly communicate with reviewers while they play their games. What would it mean if the writers and directors of films could lean over and whisper “this part is cool because of this” or “this means this, so if you want to write that down…” to movie critics? Kind of changes the meaning behind the phrase “an informed review” and moreso doesn’t exactly foster good critical thinking. Is there no, like, “Xbox Live Reviewer Safe-Zone?”

chmmr writes (August 21st, 2007 at 00:08):

Minter was definitely unnecessarily nasty about the whole thing, but what I found interesting was that the reviewer either didn’t grok or ignored the central scoring mechanic of the game, played it like it was a straight Tempest clone, and then marked it down for being such.

Minter’s games are very much not about uber skills, easy enough to lend themselves to very chill sessions. The “shit play” Minter identified wasn’t a lack of skill so much as a willful ignorance of the main game mechanic.

So while like any reviewer he is entitled to write whatever opinion he has about the game, if he very deliberately ignored a key part of the game, comparable to playing through the first few hours of GTA without driving any cars… that kind of puts things somewhere in the grey area between “idiot reviewer slags perfectly good game for no reason” and “arrogant developer deems no-one good enough to play his shitty game”.

What Minter should have done is emailed the guy, tried to understand his perspective, explain what he might have missed, and if the guy wasn’t budging made a reasoned post on his own blog about why he feels the guy is wrong.

The unaddressed issue, and something I don’t know yet not having played the game, is how well-explicated the scoring mechanic is. If Minter simply dropped the ball telling players why it’s so important, then maybe a 2 out of 10 isn’t unreasonable.

chmmr writes (August 21st, 2007 at 00:08):

Another idea in the mix is the difference between a review versus a critique. A review is supposed to tell you whether or not something is a waste of money, and for that I’m with Costikyan that “it rarely takes me an hour to decide a game sucks … or rocks”. An honest and worthy critique I would argue takes much longer – you need to really digest the game’s design and evaluate it as a creative work.

I realize of course that the controversy is over a review. I guess I just wish there were more critiques because we’re already drowning in relatively meaningless number scores and thumbs.

n0wak writes (August 21st, 2007 at 01:08):

The unaddressed issue, and something I don’t know yet not having played the game, is how well-explicated the scoring mechanic is.

Yeah, that’s the thing that we’ll see when the game is out. If he assumes that a random player is as knowledgeable of the game mechanics as his diehard forum community members, then he’s really wrong.

If the game does explain things clearly and the reviewer completely ignored it, then he might have a point… but he’s still wrong. Definitely not the way to go about it.

Alex writes (August 21st, 2007 at 09:08):

Yep, the review is somewhat ignorant. Having played it on Partnernet, SG’s definitely a game that requires every new player go through the tutorial because the rules are not easy to discern through pure play. Which could be a criticism in itself, of course. The review’s points about it being frustratingly hard to see what’s going on, what with all the psychedelics, are pretty fair, though as Minter says, it’s part of the game to learn how to read the screen for dangers. SG is a pretty uncompromising game… But agreed, Minter does need to keep himself from emotively reacting to undesirable reviews.

On reviewing in general, I find that almost every game can be basically evaluated in the first 30 minutes of play – it’s rare that I find myself revising my opinion after that time. But extensive further play is a complete necessity to understand, refine and communicate that view.

chmmr writes (August 21st, 2007 at 10:08):

The readability issue I’m totally sympathetic to, I find a given screen of Geometry Wars pretty hard to parse after the first few levels.

Aubrey writes (August 22nd, 2007 at 09:08):

The issue of readability is reduced when the systems are internalized better in the player.

A while back there was a study on fighter pilots’ training methods. Pilots who trained in abstracted, cut down simulations ended up performing better than those who did realistic simulations and took early practical experience. One conclusion from this could be that explaining a system without distracting noise (which you can’t necessarily discern as important or not) prepares people to ignore irrelevant noise later on.

Geometry Wars’ noise is not as much of a problem as Space Girraffe’s since it starts out clear, allowing the player to explore the mechanics with visual clarity. Space Girraffe starts out noisey, depriving the player – somewhat – an easy playthrough in the early stages.

I have this theory that when visual feedback is disrupted by noise/obfuscation, then the player has no choice but to play not against the game qua console, but against their own interpretation/prediction of the game state. This internalization is faster than parsing the game’s visual feedback, and thus, can occasionally result in some amazing feats of blind-play… increased ability as a result of blindness. I could totally be wrong about this.

New comments are disabled for this post.