PS3 News: Should Game Reviewers Be Reviewed? - PS3 News

Members Login: Register | Why sign up? | Forgot Password?

Should Game Reviewers Be Reviewed?

It's a well-designed April Fool's joke, but it brings up an interesting question.

With so many so-called journalists and video game critics tossing their opinions and analysis into the Metacritic and GameRankings charts, it can be difficult to pinpoint the good reviewers. Would it be easier to have a site like "MetaMetaCritic" for people to locate the best writers and those who serve the public's interests well on a frequent basis?

These days, most people just stick to a favorite source or two, but critics tend to move about, and gamers may want to know where they go. After all, some people just tend to identify with a critic's tastes and style; they say to themselves, "Well, every game this guy has recommended I've liked, so I'm going to keep going by his reviews." And besides, wouldn't a rating system of some kind add some much-needed legitimacy to the industry? Or would it have the opposite effect, in that it actually seems childish on the surface? After all, there's no site that quantifies the quality of movie or music reviewers, is there?

Still, with the vast majority of game reviews online and not in print, and the fact that just about anyone can write a review for any given site, some sort of qualifying process might not be a bad idea. It might be especially helpful for newcomers to the world of video games, and such a site could even evaluate entire websites in terms of review quality, which would make things even easier. What do you think?

Tags: game critics, game reviewers, video game journalists, gamers

4/1/2012 8:54:00 PM Ben Dutka

Put this on your webpage or blog:
Email this to a friend
Follow PSX Extreme on Twitter

Share on Twitter Share on Facebook Share on Google Share on MySpace Share on Delicious Share on Digg Share on Google Buzz Share via E-Mail Share via Tumblr Share via Posterous

Comments (34 posts)

bebestorm
Sunday, April 01, 2012 @ 9:41:20 PM
Reply

Its not a bad idea since so many rely on reviews I don't because I know what I like and don't like.

Agree with this comment 0 up, 0 down Disagree with this comment

ZettaiSeigi
Sunday, April 01, 2012 @ 9:57:42 PM
Reply

Reviews are too subjective for me to solely rely on them for what I'm going to purchase. It's not a bad idea to read them, but it's also a better thing if you choose the reviews that you read.

I am mostly impressed by the reviews here (which is why I am here in the first place) so the ones getting good feedback here usually gets to my shopping cart. It's not a matter of which review had the most hits. What counts is the review is fair and is actually something that you find yourself agreeing on.

Agree with this comment 0 up, 0 down Disagree with this comment

WorldEndsWithMe
Sunday, April 01, 2012 @ 10:08:39 PM
Reply

I think it's a good idea since so many nincompoops are employed or allowed to review, but it would probably have to be a public service of some kind. It would be hard to make any money being a critic of the critics.

Agree with this comment 2 up, 0 down Disagree with this comment

ZenChichiri
Monday, April 02, 2012 @ 12:30:23 AM

I think this is the first time I've ever read the word nincompoop in written form.

Agree with this comment 6 up, 1 down Disagree with this comment

maxpontiac
Sunday, April 01, 2012 @ 10:20:57 PM
Reply

Of course. I have said this before but now is the time for an association of reviewers to be formed.

A governing body of professional reviewers would bring instant credibility to a game review and be far more acceptable to a game developer/publisher.

Start the ball rolling Ben!

Agree with this comment 8 up, 0 down Disagree with this comment

Neo_Aeon666
Monday, April 02, 2012 @ 11:06:29 AM

Ah yeah that's an awesome idea. But the people in that *council* need to be *unbribable* ...

They need to choose the members of the association based on previous reviews. If unbiased and legitimate, they would qualify.

Then they could *label* reviews from F- to A+.

Really like your idea.

Agree with this comment 1 up, 0 down Disagree with this comment

Raze22
Monday, April 02, 2012 @ 9:42:30 PM

Also they would have to be open minded, fair, and logical...

Agree with this comment 2 up, 0 down Disagree with this comment

PC_Max
Sunday, April 01, 2012 @ 10:57:10 PM
Reply

Well, my thought is never rely on one source only. Browse around, read critic reviews and read user reviews. You should be able to get a general idea of the success of a game and whether to purchase it or not.

I am sure there was a time I would have just gone out and bought a game based on hype, but that was a long time ago and now weigh all the reviews in. :)

Critics are individuals each having their own tastes and biases. You see it movie reviews as well.

In the end you make your opinion, don't shut yourself out based on a single or 2 reviews. Get out there and see what a multitude of people are saying.:)

Keep Playing!

Agree with this comment 2 up, 0 down Disagree with this comment

maxpontiac
Sunday, April 01, 2012 @ 11:26:06 PM

I used to be like that until I came across PSXE. Once you find someone you trust, you really don't need to go elsewhere.

Agree with this comment 5 up, 1 down Disagree with this comment

packersfan66
Monday, April 02, 2012 @ 7:27:18 AM

Don't get me wrong, I love the reviews here. But you can almost guarantee that a ps3 exclusive on metacritic will have a psx rating near the top

Agree with this comment 1 up, 2 down Disagree with this comment

Underdog15
Monday, April 02, 2012 @ 8:56:05 AM

@PackersFan
While there are definitely other PS3 exclusives that this site can be found near the bottom. There are also a few in between.

And since any sane person would rate many PS3 exclusives as the cream of the crop, it shouldn't be a surprise that a PS3-centric site would be found near the top when it comes to a AAA title.

Agree with this comment 2 up, 0 down Disagree with this comment

Temjin001
Sunday, April 01, 2012 @ 11:24:13 PM
Reply

Hmm while it's an interesting idea I don't think I desire something like that much, not that I'm opposed to it or nothing.

Typically, if I know a game is coming from a respected studio or director I don't even need a review to validate my purchase. Anymore it's the more obscure or lesser known productions which receive rave critical response, like Journey, where I'm attentive to a game's identity. Critics being the guinea pig to have played something lesser known and uncertain. And outside of frequenting here, I usually don't really pay too much attention to specific editors outside of metacritic averages.
I don't need a review to read for AC3, ME3 etc, or even NG3. Heck, despite NG3's poor critical response I've still played and enjoyed it much, validating the value of my time and money, contrary to what IGN would try to have me believe. My next money is on DOA5, and the demo easily confirmed it as D1P. So too will I get TTT2 based on the fact that Harada is still the man at the helm and I always love Tekken.
Really, anymore, I actually have a hard time even taking goty awards or any sort of heirarchy of eliteness with much more than a generalized level of knowing how much production has gone into the highest marked games. It seems if any game received tons of budget funding, it's probably going to get high reviews. If it hasn't, we'll, presentational elements suffer but the game may still be great and valuable. Except for download titles, those seem to be much more innovation-centric.
But I still prefer Vanquish to Gears, even though Gears probably cost more.
I still prefer NG to GoW, even though GoW cost more.
All are worth playing of course. So seeing that I would take Infamous 2 over Uncharted 3, even though U3 was probably way more expensive and earned way higher ratings. All I really need from a critic is a thumb up or thumb down and I'll decide for myself how much I enjoy it at home. Because that's really how it seems to happen anyway for me anymore.

Sorry, this went sort of off topic some =p




Last edited by Temjin001 on 4/1/2012 11:28:05 PM

Agree with this comment 2 up, 0 down Disagree with this comment

Palpatations911
Monday, April 02, 2012 @ 1:40:06 AM

I think X-Play uses a similar system on their reviews.

Agree with this comment 0 up, 0 down Disagree with this comment

Veitsknight
Monday, April 02, 2012 @ 1:18:26 AM
Reply

I think that the 5-star rating is better than the 10-point rating system. A game that would normally be bashed by gamers for being 7-point-something while in a 5-star system, it could get 3/5 stars and still look
decent.

Agree with this comment 0 up, 0 down Disagree with this comment

killerbee12
Monday, April 02, 2012 @ 1:30:24 AM
Reply

In some ways I think you should be reviewed.
And edge magazine..

Agree with this comment 0 up, 4 down Disagree with this comment

ZenChichiri
Monday, April 02, 2012 @ 1:39:39 AM
Reply

It sounds like a good idea at first, but once again the quality of those reviewers of reviewers could have the same problem as reviewers do (because we're all human). Then you're going to need reviewers for the reviewers of the reviewers. What an endless abyss that could turn out to be.

If people didn't take reviews as gospel, then we wouldn't have a problem with some bad reviews in the mix because we can think for ourselves. Unfortunately there are many people that will take a look at review scores instead of their own personal preferences. Objective analysis of games is all good for looking at the quantitative side of things, but we all have individual preferences about specific genres.

I don't think it's the reviewers we need to worry about, but instead the ability of people to interpret them with regards to their own personal gaming tastes.

Last edited by ZenChichiri on 4/2/2012 1:40:39 AM

Agree with this comment 3 up, 0 down Disagree with this comment

Lawless SXE
Monday, April 02, 2012 @ 2:12:37 AM
Reply

Quis custodiet ipsos custodes?

I think that it could be a good idea if it was implemented and utilised correctly. A select group of reviewers of a high calibre of quality selected by a third party to go over each individual critic/site and giving them a score based on the quality of their work could go a long way towards the legitimisation of many reviewers, as well as weeding out those that are unbalanced, unfair or just plain biassed. It's not necessary for films because not everyone thinks that they're entitled to review them on a professional basis, but games are an entirely different field for some bizarre reason.
Peace.

Agree with this comment 2 up, 0 down Disagree with this comment

___________
Monday, April 02, 2012 @ 4:45:50 AM
Reply

f*cking oath they should!
problem is every man and his dog can setup a website these days and post BS reviews!
i mean im sorry but how the f*ck does ninja gaiden 3 deserve a 8.5/10?
i mean come on!
if that deserves a 8.5 then every other game this gen deserves a perfect score!
thats exactly why theres so many fluctuating reviews these days, you honestly have a better chance of picking the lottery then telling if a games decent or not!
i stopped holding any weight in reviews when GTAIV got perfect scores.
a taxi driver game with a GTA name on it deserves a perfect score?
really?
wow, R*s bribes are THAT good are they!?

Agree with this comment 1 up, 2 down Disagree with this comment

Lawless SXE
Monday, April 02, 2012 @ 6:01:42 AM

Incidentally, a Taxi Driver game could be absolutely fantastic if done properly. Travis Bickle is pure awesome.

Agree with this comment 0 up, 0 down Disagree with this comment

___________
Monday, April 02, 2012 @ 8:43:08 AM

i dunno about that.
tbh i cant think of anything more boring then just carefully driving passengers around the city!

Agree with this comment 0 up, 0 down Disagree with this comment

nogoat23
Monday, April 02, 2012 @ 6:17:01 AM
Reply

I haven't read through the comments, shame on me, but I don't think there should be a meta meta critic. If we go to the second meta, then why not add a third, or a fourth? Who's going to review the reviewers of the reviewers of the reviewers??

No, we should just use our own judgement to decide for ourselves who is a credible reviewer and who isn't, just like we judge for ourselves who is a credible news source and who isn't.

Agree with this comment 0 up, 0 down Disagree with this comment

Ishkur
Monday, April 02, 2012 @ 6:59:38 AM
Reply

No. That would make the reviewers go for that score, instead of being honest (top rated journalists getting money for high scores?). Simple as that. A system where it would work probably exists, but not in the form of a simple scale. It'd have to scale every genre of games (which that reviewer likes/is good at), as well as the professionalism of his/her reviews. It's too much imo. I'd love to see something that would work though.

Agree with this comment 0 up, 2 down Disagree with this comment

Ben Dutka PSXE [Administrator]
Monday, April 02, 2012 @ 9:51:44 AM

Uh...you're not getting the point. They'd be rated on the quality of their reviews. Scores would be irrelevant.

Agree with this comment 3 up, 0 down Disagree with this comment

Ishkur
Thursday, April 05, 2012 @ 3:24:33 AM

No, I'm aware of that, what I mean is this:
People would look for the highest rated reviewers for their games through this metametacritic, thus creating an interest for publishers/developers to funnel money to said reviewers for higher scores, that's all. The score the reviewers will go for, is the one for themselves is what I mean as well. Think we mistook each other there :)

Agree with this comment 0 up, 0 down Disagree with this comment

Dante399
Monday, April 02, 2012 @ 8:30:48 AM
Reply

That's the most logical thing I have ever heard in Video Game industry. Reviewers have to be reviewed in order to reach a higher level of credibility.

Agree with this comment 1 up, 0 down Disagree with this comment

Underdog15
Monday, April 02, 2012 @ 8:53:34 AM
Reply

I definitely think there needs to be a governing body or evaluation of reviewers as there is too much free reign without guiding principles in the industry.

However, allowing users to rate reviewers would more likely just batter down hard-working reviewers than help. People would be more likely to bash a reviewer than write a good review. You wouldn't have any reviewer scoring higher than a 5/10! lol

MetaCritic is nice because MOST reviewers are professional, so you can get an accurate average of games. But a meta-metaCritic would serve as little more than a "complaints department".

Something far more structured would be necessary to take something like that seriously.

Last edited by Underdog15 on 4/2/2012 8:54:03 AM

Agree with this comment 2 up, 1 down Disagree with this comment

Highlander
Monday, April 02, 2012 @ 11:10:13 AM

Most....

Metacritic's problem is that the source of some of the reviews that are included in their scoring are not particularly professional sources. I think that they include too many review sites without really vetting the quality or qualification of the site. That and their algorithm for generating the metascore seems overly sensitive to outliers on the lower end of the scores.

Agree with this comment 0 up, 0 down Disagree with this comment

Highlander
Monday, April 02, 2012 @ 10:41:24 AM
Reply

This is actually a really good idea. 360 degree review - so to speak. Why not review the reviewer?

Of course the problem is that if gamers are reviewing reviewers, there is always the possibility of a popularity contest or a witch hunt. So, I'm not sure how that would work out, but it might be interesting if a panel of a dozen or so different game news sites could review each other.

Agree with this comment 0 up, 0 down Disagree with this comment

Underdog15
Monday, April 02, 2012 @ 10:48:03 AM

Might be a good idea. Then you could remove the top 2 and bottom 2 scores a la Olympic judging style!

Agree with this comment 0 up, 0 down Disagree with this comment

Highlander
Monday, April 02, 2012 @ 11:08:16 AM

That's a good idea. Stripping outliers is always a good idea.

Of course I have wished that Metacritic would re-evaluate their algorithm for some time for the same reason. Low score outliers seem to have a disproportionate effect on the metascores.

Agree with this comment 0 up, 0 down Disagree with this comment

Twistedfloyd
Monday, April 02, 2012 @ 12:38:50 PM
Reply

It's a good idea I think. I always look at Ben's reviews as they're really well written and I agree with a lot that he says about many a game. Same with Colin Moriarity at IGN and some people at Joystiq. It'd be cool to see who users says are the most credible game critics, but at the same time, I don't think it's necessary. I know who I trust when it comes to game criticism.

Agree with this comment 0 up, 0 down Disagree with this comment

Nynja
Monday, April 02, 2012 @ 2:48:21 PM
Reply

Ben,

Does this mean you have plans on reviewing the critics?

You know, that would be one hell of a read! And project!

You'd have to rate them on:

"Gaming Wisdom" - Knowledge of the industry and gaming as a whole
"Thoroughness" - Game completion prior to review
"Spoilers" - Ability to review without spoiling
"Favoritism" - Amount of favoritism shown to competition / console
"Presentation" - Ease of reading, grammar, relative descriptions of game play, etc

Then average out their scores.

Agree with this comment 1 up, 0 down Disagree with this comment

Rogueagent01
Monday, April 02, 2012 @ 9:14:01 PM
Reply

April fools or not this is a good idea! It's a watchdog system which already exists in many other industries(especially those involving money). I rarely read reviews myself as I already know what I will like, but this would truely benefit those that do use them to judge purchases.

I would think that the reviewers of the reviewers would need to write up a huge Bio of themselves. So that the people that chose to read the reviews would know how and why they are grading the way they are. So long as they have their Bio and a boss reads their reviews and ensures consistency then this could easily manage itself. I don't see how it could become a witch hunt as the reviews would be calling out bad journalism/favoritism/bias, all of which should be called out in the first place.

I already wish that game critics would do this themselves as the things they look for in a game might not be what I look for in a game, hence the reason I can't stand the majority of critics. Ben that is why I have read as many reviews as I have from you as you seem to look for a lot of the things that matter to me. And rather then throwing a game under the bus like most critics do, you will actually use similar games to reference why someone like me might still want to buy it, I love that about you.

Agree with this comment 0 up, 0 down Disagree with this comment

trumpetmon65
Tuesday, April 03, 2012 @ 4:18:05 PM
Reply

meh. We have demos to help us try out games and sony's one hour full access demo is also a great way to try out the new stuff., Also Gamestop has that 7 day no questions asked return policy for their used games. If you want to try the game out before you buy it new (to help the makers out)just return it and then buy the new copy. Boom, problem solved. So I say there are plenty of safety nets for us out there.

Agree with this comment 0 up, 0 down Disagree with this comment

Leave a Comment

Please login or register to leave a comment.

Our Poll

Will the PS4 continue to outsell the Xbox One?
Definitely. Microsoft has no hope.
Probably, but it will be a tight race.
I doubt it; Xbox One will come back.
No, PS4 will falter over the years.

Previous Poll Results