Forums » Suggestions

Peer review system for player submitted missions?

Feb 07, 2006 Phaserlight link
Without knowing too much about the mission editor except that the devs want to make it available to the playerbase to a certain extent, would it be useful to implement some kind of peer review system?

A) you want to maximize content generated, and
B) you want to insure the quality of said content.

If we start getting hundreds of player submitted missions per week, the devs aren't going to have enough time to sift through all that.

What about having some kind of simple peer review system where once a mission (or mission tree) is submitted it becomes available for review, either on the test server or as text on a website. Players can then vote 'keep' or 'kill' based on the mission's continuity with the Vendetta universe. If after two weeks of voting the mission (or mission tree) passes with a 2/3rds majority of 'keep' votes, it gets sent to the devs for further review and implementation.

Of course the devs would have the final say in any player submitted mission... perhaps a mission submission would get killed in peer review but the devs happen to like it and decide to implement it anyway.

At least with a peer review system there would be an easy way for the devs to keep track of which missions are popular amongst the community, and it would be a useful tool for sorting the quality missions from potential 'omg! kill 10 arklans cuz I say so lolz!!1!' missions.
Feb 07, 2006 LeberMac link
Some unliked players will never get their missions accepted thru peer review, even if they are the best missions ever. Imagine Blackhole Goldclaw or genka submitting missions.

And... PLEASE... build in a spell-checker so Lord Q and Mecha's missions can be understood. Heh.

And yes, after the peer review, the devs or guides should take a look at it and decide whether to implement it or not. In fact, that might be a great role for a part-time employee or existing guide to take on full-time. Content is what this game needs BADLY. BADLY! Assigning one person (that Bergman trusts) to be in charge of new "content" would help out a good deal.
Feb 07, 2006 Phaserlight link
Make the submissions anonymous. That way they will be judged on content alone.
Feb 07, 2006 LeberMac link
*ding*

Phaserlight makes another outstanding point. Yes. Anonymous missions. Good.
Feb 07, 2006 LeberMac link
Oh, how about artwork that goes with the mission screens? Can that be done?
For example, for LeberMac's Tequila Rescue Mission, have a picture of the missing tequila bottle and the last known wherebouts shown on a map. Images are worth 1,000 words. Makes sense to allow images to be incorporated into the mission stages/messages.
Feb 07, 2006 Whistler link
Something like this is already being planned, though I don't think anonymity has been considered. I think that there will be a few layers of review and that good missions from unpopular players would get a fair shake. The Devs are more savvy to VO politics than people realize. I don't know that special artwork has been considered either - though that gets kinda tricky with permissions and control over the look of the game.

Spelling is always important, which is why the station news is generally looked over pretty carefully prior to publishing. I suspect that there will be pretty careful quality control both in terms of errors and potential misuse.
Feb 07, 2006 Renegade xxRIPxx link
heh misuse as in the bp missions :D
Feb 07, 2006 ananzi link
im against this because it should be all or nothing.

either all democratic, where devs are just another 'voter', or
not democratic (like it is now).

sure you can have people who have final say in the missions, but they should be rotating not the same person all the time.

------

anonymousness is brilliant idea.

------

believe me, i was on kuro5hin for several years and the system broke down somewhere between 'k5 is free' and 'rusty had a bad day and decided to ban a bunch of people and not give any reason'
Feb 08, 2006 toshiro link
I agree with ananzi that the people who have the final say should be rotated (perhaps voted from the playerbase?) periodically. Also, it should not be one single person to have the final say, but perhaps a group of 3 people (2/3 majority).

I am also rather for the more or less democratic way of peer reviewing, where the developers may have an increased voting 'weight' (say, 5 votes instead of 1 per developer), since, after all, it is their game, and we do not want to vie for control over it with them too much (a peer review system would exactly do that, namely reduce control of the developers over the evolution of the game, in my opinion).
Feb 08, 2006 Renegade xxRIPxx link
devs don't need votes, if they say its in then it is in they have patriarchical rights. Maybe only thing that needs to be done for the devs votes is a majority for them, abstaining = vote to yes.

and the people involved in the voting should be the same people that are guides now. or then.

But the anonimousity is a good feature.
Feb 08, 2006 johnhawl218 link
Anonymous Entries should be a must

I don't really think there should be any peer review. If anything guide/devs should do the review and decide to keep, reject or modify suggestions as they see fit with devs getting final say. Even if this slows down the implementation of the missions themselves.
Feb 08, 2006 LeberMac link
No I tend to agree, this isn't a democracy. Let the people who have the "vision" (devs or longtime guides) approve/modify/reject the missions as they are submitted, with at least a little bit of explanation if they are rejected.
Feb 08, 2006 zamzx zik link
meh, I am not really" hated" I just pissed a few people off.

As far as I know, the guides like me perfectly fine (no more or less then anyone else)
same with the devs.

I think that anonmynisty is good if we (the players" are voting/grading the mission, but once it is released, it should have a name on it (so people want to make missions)

also, the guides and devs should know who is submitting it, so if someone spams the mission stuff, they can ban/mute them.
Feb 08, 2006 terribleCabbage link
I'm with Johnhawl and Lebermac. Have the players just whittle down the list of choices, and let the devs have final say on which missions end up being implemented.

- Tol Kerhys
Feb 08, 2006 LostCommander link
Wonderful, Phaserlight and LeberMac!
*Stamp of Approval*
Feb 09, 2006 Phaserlight link
For a good example of a peer review system, check out the urban dead suggestion wiki:
http://wiki.urbandead.com/index.php/Suggestions