PS3 News: Yeah, See, This Is Why You Can't Really Trust User Reviews - PS3 News

Members Login: Register | Why sign up? | Forgot Password?

Yeah, See, This Is Why You Can't Really Trust User Reviews

For the record, here at PSXE, we get a lot of really great user reviews. That says a lot about the community and at the same time, it proves that plenty of gamers are fully capable of providing their peers with accurate, informed breakdowns.

But for the most part, in general, I would not recommend taking user reviews as gospel. And I certainly wouldn't recommend taking their word over the word of professional critics. That may sound elitist but in reality, it's just a fact. There is a certain level of knowledge and expertise required, regardless of what gamers want to believe, and I address that to some extent in this forum topic, if you're interested.

Anybody can write a review of anything. Just because I can review games doesn't mean I can review music (at least, not nearly as well). I wouldn't expect anyone with a brain to take my opinion on a new album over the opinion of a Rolling Stone veteran. And sadly, when it comes to this industry, when there appear to be more immature individuals than ever before, it's almost never a good idea to trust random user reviews. Take the user review score at Metacritic for the recently released Call of Duty: Black Ops II: The average score is a 4.0. I mean...come on.

I'm fairly certain the game is better than a 4. The critic average sits at about an 8.5, which seems about right. But when you leave it up to individuals who clearly have only one goal - and that goal has nothing to do with assisting consumers - and that goal is essentially the mindset of a 5-year-old, the results are less than reliable. And that's putting it mildly. Some people out there can write great reviews but you know what? It's quite clearly a very tiny minority, as it is with just about anything else. I'm sure you think you can write a review every bit as good as one Roger Ebert pens, but FYI...you probably can't.

Tags: metacritic, user reviews, call of duty black ops ii, black ops, bo2 user reviews

11/14/2012 11:48:29 AM Ben Dutka

Put this on your webpage or blog:
Email this to a friend
Follow PSX Extreme on Twitter

Share on Twitter Share on Facebook Share on Google Share on MySpace Share on Delicious Share on Digg Share on Google Buzz Share via E-Mail Share via Tumblr Share via Posterous

Comments (23 posts)

Highlander
Wednesday, November 14, 2012 @ 12:16:00 PM
Reply

lol, more to the point, this is why you can't trust Metacritic user review scores... ;)

Metacritic's problem is that it takes all comments, scores and reviews at face value. That's why I prefer not to look at the actual metascore from user reviews, but rather their distribution. When you see a review score distribution something like this;

5 ********* (30%)
4 ************* (40%)
3 ***** (18%)
2 ** (6%)
1 * (3%)
0 (0%)

You can be pretty sure that the game is a solid 4, or perhaps a 4.5. But when you see;

5 ****** (15%)
4 ************* (33%)
3 **** (10%)
2 ** (5%)
1 * (3%)
0 ************ (30%)

You can be certain that some gamers are deliberately trolling the user review scores, and it's pretty easy to tell that the game is still probably a solid 4/4.5 game that has attracted a few haters.

The actual distribution will vary, but any time I see a huge spike in scores of 1 or 0 that doesn't fit the rest of the distribution, I know that the low scores are unmerited.

Of course you might see something more like this;

5 (0%)
4 * (3%)
3 ** (6%)
2 ***** (18%)
1 ************* (40%)
0 ********* (30%)

That would be a fairly reliable indicator that the game is crap.

It's all about the distribution of scores.

Agree with this comment 4 up, 0 down Disagree with this comment

matt99
Wednesday, November 14, 2012 @ 12:20:32 PM
Reply

There are three reasons you should trust professional critics over the average joe: 1) they have professional integrity (most of them anyways) 2) they have a reputation to keep and 3) if their reviews aren't reliable people don't read them and they don't make any money!

Agree with this comment 0 up, 1 down Disagree with this comment

Doppel
Wednesday, November 14, 2012 @ 12:40:58 PM
Reply

I would point fingers at some "professional" reviewers intentionally downranking a game (or a system, see the PS3 and PSV fiasco) to attract views and controversy and all that crap.

Actually, I'll name names: Kotaku.

Agree with this comment 4 up, 0 down Disagree with this comment

dillonthebunny
Wednesday, November 14, 2012 @ 12:43:59 PM
Reply

the kids that leave reviews are using Metacritic as a way of bringing down the scores, they have been doing this for years. as I said in another conversation.. its these skirmishes from other sites/ consoles where they feel they have to troll, and Metacritic lets them do it.. its a battlefield out there, lol.

personally I feel if Metacritic took it away my life would not be impacted what so ever.

Agree with this comment 1 up, 0 down Disagree with this comment

dillonthebunny
Wednesday, November 14, 2012 @ 12:46:21 PM
Reply

yep Doppel, The Edge has been doing that for years, and a few sites too.... I dont think that will ever change, sadly.

Agree with this comment 1 up, 0 down Disagree with this comment

duomaxwell007
Wednesday, November 14, 2012 @ 1:17:27 PM
Reply

and critic reviews are better? gamespot game RE6 a 4.5 after all

Agree with this comment 1 up, 2 down Disagree with this comment

Ben Dutka PSXE [Administrator]
Wednesday, November 14, 2012 @ 5:27:06 PM

"and critic reviews are better?"

Yes.

Agree with this comment 3 up, 1 down Disagree with this comment

Underdog15
Wednesday, November 14, 2012 @ 6:08:47 PM

usually, yes

Agree with this comment 3 up, 1 down Disagree with this comment

cLoudou
Wednesday, November 14, 2012 @ 1:43:33 PM
Reply

You just need to look at the critics reviews on the other side. As much as I dislike COD I've never said its a bad game/series. I just hate everything about it.

Agree with this comment 1 up, 0 down Disagree with this comment

Knightzane
Wednesday, November 14, 2012 @ 2:42:56 PM
Reply

Thats because critics dont like being criticized. Some of the worlds biggest hypocrites are critics. Seriously, Korn's Untitled was "To different" Then Path to Totality releases and the exact same person said "Oh this is the album of the year! Anyway, I trust my own judgement on a game. I just wish that some people didn't miss out on some games based on a reviewers opinion. Anyone here play silent hill downpour except me? Or even white knight chronicles 2? No? Anyone? Thought so.

Also because i know theres smart arses on here, i realize i used a music critic as a reference but thats only because i don't look into names of game critics. The only critics name i know is ben and thats only because... well... He's ben. I think ben is the only critic i can actually respect because he does try to give a game a true score. I have also noticed tons of contradictions but they are still legit compared to every other critic. So no offense to ben, but he is the best critic and thats why i like this site so much. Oh, and the people who come here seem cool too. Well, most anyway... __________

Agree with this comment 3 up, 4 down Disagree with this comment

Beamboom
Wednesday, November 14, 2012 @ 3:02:21 PM
Reply

Oh man don't get me started. User ratings are the ultimate waste of bandwidth, and game ratings the worst of all user ratings again.

Agree with this comment 7 up, 0 down Disagree with this comment

Clamedeus
Wednesday, November 14, 2012 @ 9:45:53 PM

Agree completely.

Agree with this comment 2 up, 0 down Disagree with this comment

Russell Burrows
Wednesday, November 14, 2012 @ 3:24:23 PM
Reply

Agreed Beamboom.

However a game like Black Ops 2 thats just out now and in one play session does 16 disconnects/boots from server does deserve a downgrade since it still needs patching to perform better.

I hate games that are released halfbaked i.e. Bethesda aka Befreezeda or Bugtheseda.

Agree with this comment 5 up, 0 down Disagree with this comment

xenris
Wednesday, November 14, 2012 @ 3:24:45 PM
Reply

I look at user reviews in a certain light. I think the score has truth to it but you have to do some detective work.

For example look at Dragon Age 2. Everyone who loved DA1 that I know thought DA2 was a waste of time and a complete joke, myself included. But the critic review on Metacritic was and maybe still is around 8 I think?

In that case the user reviews much better represented what the game was like, and they had a less biased perhaps bought opinion. To this day I still don't know how that game got the scores it did, I mean I have an idea and it rhymes with ash honey.

As for the Black Ops 2 user score, I know most of it is trolling but I think it represents some truth as well. People are fed up with the franchise, and likewise are fed up that it seems to get free passes every year for not really trying that hard.

Call of Duty is something of an enigma, I have no idea how it gets the scores it does, year after year. Its like not ONE professional game reviewer doesn't like CoD? That seems suspicious when I know a lot of people who don't both in real life, on youtube and on message boards.

I have been playing the single player, and certain issues that people hated that were in warfighter are in BO2 but I haven't seen one reviewer bring them up they just pass over them. Maybe they are hidden better in BO2 but they are still there, things like if you try to run ahead or go where the game doesn't want you you die, enemies acting much like shooting galleries targets, waiting for your team at checkpoints where they move something or lift you up to progress the level etc.

I like BO2, much more than I thought but it is not innovative, not revolutionary and lacks some polish in certain areas. I would give it a 7 at most and remember that is a good score. Problem is no one is giving that, all the professional reviews are saying whoooaa CoD reinvented itself bla bla bla, and it didn't except for the new coat of paint. Multiplayer plays the same, and the same issues that have been there since MW1 are STILL there again no one cares.

Anyway I am interested to see what Ben gives this game.

Last edited by xenris on 11/14/2012 3:25:51 PM

Agree with this comment 2 up, 0 down Disagree with this comment

bigrailer19
Wednesday, November 14, 2012 @ 10:52:17 PM

I don't think the user reviews necessarily represent "what the game was like" better. The idea here is that, you have a game in DA:Origins which went over well. The fans were disappointed in DA:2, but it's still a good game. It's easy to mask that with frustration stemming from something that's distant from what you liked about the first. The critic reviews as they should scored it on what it brought to the table as a video game. Not in regards to how good the first was. Although it should (as for any game) be held to that level of quality.

It's like the FF franchise. There's no doubt that FF13 is a good game, but it's not a FF game, so the fans are upset.

RE, Also a case of this. RE5 is a great game, RE6 is a great game, but the only thing closely resembling a RE game in either is the Leon campaign in RE6. Naturally fans are upset. Apparently with RE6 though so are the critics.

So no, the user reviews have bias, the critic reviews do not.

And for the record this happens to me all the time. Games change for the masses and I can't stand it! Socom did and it's frustrating to me!


Agree with this comment 2 up, 1 down Disagree with this comment

xenris
Wednesday, November 14, 2012 @ 11:18:36 PM

Well I completely disagree with DA2 being a good game. I think it coasted by on its decent production values and the fact that it was a Bioware game.

If it didn't have bioware on the title I think we would have seen different scores. It reused so many assets, had a tiny world, hardly any loot, and the story was not really bioware quality in my opinion. Even on its own without DA:O to be compared to, DA2 for me was a mediocre game, and I strongly believe it only got by because of the name.

All I meant big railer was that the user reviews reflected some truth to how the game was received by fans of the series which if you sift through the troll ones you would see if you would or wouldnt like it.

Agree with this comment 0 up, 0 down Disagree with this comment

Jed
Wednesday, November 14, 2012 @ 4:45:37 PM
Reply

Here's a good one,

"scorpionm84

Multiplayer is egual to two last year. Notthing change is a simpe Copy/Paste with some change in menu graphic. Bad work. If you can safe your money for another FPS later."

He gave it a 2

Agree with this comment 0 up, 0 down Disagree with this comment

ulsterscot
Wednesday, November 14, 2012 @ 5:56:34 PM
Reply

MOH Warfighter was an 8 in my book - BLOPS is a 9 - 9.5 - but then im an FPS Loyalist.

The Metacritic average was likely from mostly JRPG fans ....

Agree with this comment 0 up, 3 down Disagree with this comment

WorldEndsWithMe
Wednesday, November 14, 2012 @ 6:00:59 PM
Reply

But you can easily tell if a user review is useful or a hate score by reading the first sentence.

Agree with this comment 2 up, 0 down Disagree with this comment

Temjin001
Wednesday, November 14, 2012 @ 6:32:51 PM

yep, when it comes to user reviews it's pretty evident who's opinion seems to be a reliable one just by tone and flawed reasoning alone. I think the reviews that are terrible scores written by haters are nowhere near as problematic with me as the more uncommon critic reviews that unfairly score a game while they use their professional writing skills to persuade you that they're right on the matter. Because in this instance, albeit uncommon, they may actually convince people to avoid a game they may otherwise have enjoyed.

Agree with this comment 2 up, 0 down Disagree with this comment

Jed
Wednesday, November 14, 2012 @ 8:33:17 PM

Most of the user reviews on that page weren't much longer than one sentence.

Agree with this comment 1 up, 0 down Disagree with this comment

Gamer46
Wednesday, November 14, 2012 @ 9:41:36 PM
Reply

I'm not a big CoD fan but there's no way this game is worse than a 7. More evidence that Metacritic's user scores are not to be taken seriously for anything. Halo 4 has an average score of 6.2, which is an absolute joke as that game is great, and Mass Effect 3 last time I checked was hovering around 5 which is also totally stupid. I'm not sure on this but I'd imagine if you go to IGN or Gamespot or other more well known game websites, these games have higher user averages because real gamers, not idiotic haters are the ones who frequent those places.

Last edited by Gamer46 on 11/14/2012 9:43:47 PM

Agree with this comment 0 up, 2 down Disagree with this comment

Raze22
Wednesday, November 14, 2012 @ 10:30:12 PM
Reply

Word of advice, just "trust" in your judgement. The best shopper takes into consideration customer reviews, professional reviews, and the products description. Neither one is more reliable than the other if not more equal. This method can be applied to almost anything. Got to love the internet.

Agree with this comment 0 up, 0 down Disagree with this comment

Leave a Comment

Please login or register to leave a comment.

Our Poll

The PS4 exclusive(s) reveal in December will be...
MEGATON! Biggest thing evah!
Pretty great, but not mind-blowing.
Something decent but that's it.
A waste of hype.

Previous Poll Results