Quoted verbatim from the "What were you thinking with the Tung Maud" thread in the General forum. Thoughts, comments? I'm at work, so I'll try to bounce in here every little bit to fix the post up:

"Outbursts like these make me think the devs should back off public testing of the ships. Fire up the old 18 sector universe (or hell, just a two sector universe so wormhole scenarios could be tested) and have sign ups for testers. Make it known that testers would have to be on at x day at y time for z hours. Then snag either the top ten candidates (or a randomly selected top ten), and let them test the new ships. Make it clear that this is a test server, and with only ten people to watch over (maybe stick a guide in there), it shouldn't be too hard to keep them on task.

Or take a page out of planetside's book and have a publicly available test server with limited slots and a smaller universe that is essentially a sandbox for dev changes. At the end of the day the devs could upload whatever changes they had made and let the players have at them. After a few weeks or when the gross faults are ironed out, release it to the public and patch away anything that slipped through.

This way you avoid the recurring problem of putting out unbalanced ships and weapons, only to realize the mistake a week or two later and have to yank them just as people have finished the weeks of work necessary to fly or use them. I know the devs are only four people, I know this isn't a finished game, I know, I know, I KNOW - but people are paying now. They expect a game, not a testbed. The patch notes for the last update are a perfect example: "- New trade mission available for testing"

People aren't paying to TEST, they're paying to PLAY. Ever wonder why the active players graph hasn't gone above 100 for a while? They don't like working towards something only to find that when they get there it's been changed from what they wanted. Keep changes like that seperate from the public servers.

That way, when you release a comprehensive ship balance patch, or new mission patch, in a month or a month and a half, EVERYONE gets the same thing forever. Right now stuff gets released, is exploitable for a little bit in some way, some users get to use it or even get used to it, and then it gets fixed. The devs aren't going to wipe what those that came before did, but the fact remains that for those that come after it's a different, usually more tedious, game. The advanced combat mission comes to mind.

Either way you anger some, but with a seperate test server it's all at once instead of each week a new uproar of outrage. We understand that it's a work in progress, but I think there's enough evidence that that understanding only goes so far."

Thank You for this post Star!!!!!!

that is a good idea.

i'd volunteer to test new stuff (when i had the time that is).

And once again* i can say from experience that nothing frustrates players more than a continualy changing game. There isn't so much of a problem with it in beta testing because everyone there knows that the game will be changed drasticly and repetedly, but in a released game people do expect a level of consistancy to the game universe.

*if for some reason anyone has followed my posts obsesively you will realize this is the third time i have spoken from experience as an amature game desinger the fact that i typed "once again" may not make that much sence with out this knowlege, and instead of rewording it i decided to write this needleslyt long explenation that is now explaining itself

I'd definitely help test, too. Please implement this! No more busbuggers!
-:sigma.SB

This would be a good idea if the player base was bigger, but at this point making another server just dilutes the main server:

- If the Test server is large and has many testers, then there are that many fewer players on the main server.

- If the Test server is small and has few testers, few problems will be properly discovered. (I say "properly discovered" because one or two players claiming that something is nerfed doesn't always apply to the whole game.)

(A small server and many players or vice versa isn't a valid test, so I won't list them.)

Klox you didn't read this part:

"Make it known that testers would have to be on at x day at y time for z hours. Then snag either the top ten candidates (or a randomly selected top ten), and let them test the new ships."

The suggestion is to have specific test times with a specific group of people. I do think 10 is too many for the current player base, though. Five is sufficient. (Think about how many testers there are now.)

Actually, I'd bet the biggest hurdle to overcome is the reliability of the testers. It would require investment in training the testers to give useful feedback, and that would be wasted expense if said volunteers didn't feel like testing at the given time, or abandoned testing altogether.

I'm assuming that the devs have test cases. However, we often hear about how they are four guys not a full dev group - ok. So you make the test server, put five or ten testers on it, and give them the test cases. It would get boring fast, but you wouldn't just put anyone on there. The same screening you put the guides through would apply.

Really, what are the bugs people discover? Mostly balance related - "The Corvus vult drains at 90!!? I can't get anywhere with it!" and that's the sort of thing 5 or 10 dedicated testers would discover pretty quickly. Or something like what caused them to yank the trade missions - "Hmm, I just got an insane amount of XP from this mission"

Run them through a few CtC runs. The testers would try different ships - the Centurion cargo/infiniboost thing would have been apparent very quickly. It only took players a few hours after the update to hit on that one.

Tester reliability would be less of an issue if you chose dedicated testers. FM and the guides prove that people are willing and able to responsibly dedicate time and effort for free. You don't need a professional test report, you just need "Hey, I was able to do x with y and it doesn't seem right because of z"

There would still have to be live balance tweaks, but what this would hopefully avoid are the huge ship killing problems and changes that cause everyone to freak out.

I agree this does seem like a good idea. It'd also reduce the amount of time the devs would need to spend testing things themselves.

best idea would to use some of the paying customers by making it as some sort of a 'bonus'. They get to see the new stuff first, and can brag to their friends :D

They'd actually probably want people who wouldn't talk about it. Just get in, run some test cases, run some ctc runs, have a half hour of free fighting or sector races to see whats too fast or two slow, then leave and PLEASE don't tell everyone how messed up X is or that they should level Axia because of Y. If they wanted to give a reward it might be a discount on the monthly fee of the testers or something along those lines - but keep in mind, the testers would have to earn that discount. This wouldn't be like the old days of the alpha where people would do whatever, it would be a fairly regimented, time restricted thing.

Starfisher is right. The devs wouldn't want the testers saying too much. for one thing just because it was in the test doesn't mean it will ever make it into the actual game. The idea is to have a way to catch criticle errors that can only be found by playing, before they get to the general public.

i envision some sort of in game reward for testers, based on how manny tests they particapate in (of coarse there would be some sort of screening process and known trouble-makers or lowdmouths could be banned). Perhaps there could be a discount on trade goods, like there is for mentoring.

> Perhaps there could be a discount on trade goods, like there is
> for mentoring.
It's a hoax, no buy/sell prices are altered by the 5 mentor point badge. :(

It would be nice to have a "tester" badge, like the alpha and beta badges people already have.
-:sigma.SB

well i never did any mentoring so i don't know other than what it says about it on the loading screen, but i agree a badge would be nice

I just laugh when I see alpha/beta tester badges, they mean nothing to me other then a bloated ego.

What? Alpha/beta tester badges just mean you created an account before release. They have nothing to do with your ego. How is that even relevant to the thread? Please, I'm hoping a dev sees this and posts his thoughts - it seems eminently sensible, but then I'm not a dev. Maybe there's something I'm missing. Let's not get this locked with needless flaming.

I believe he's stating that alpha testers tend to have larger egos. True, many of the people with large egos were alpha testers. The reverse, however, is not necessarily true.

(Full disclaimer: I'm an alpha tester.)

Also, his post was in response to Solra Bizna's, who mentioned that it would be nice to have a "tester" badge (for a testing server), akin to those owned by alpha and beta testers.

I agree with the idea of a test server. There are some complications.. like literally having the hardware to support it (even on a small scale) and linking it to the player database without back-propogating any changes from the test "universe". We've been through this with our own internal test server (yes, we have one, but since it's hideously non-functional due to code changes most of the time, we can't expose that very easily, although we originally hoped to). Then there's the fact that testers might need to have two copies of the game installed.. but I suppose that's not too terrible of a burden.

The other issue is.. I know our game isn't 100% by any means, and although not all the changes we make are ideal, I think it does count for something to show people that we are at least trying to move forward to a bigger, better, cooler game. If we removed this, only exposed new stuff to the "test" server and then roll out to production every month or two.. I think people might drift away from a perception of inactivity.

Live / production updates are a double edged sword, and I certainly don't enjoy the explosions of irritation and angst from making what I think are decent changes in the big picture.. anyway, yeah, we'll look at what we can do. Ugh.

Ugh indeed. I think people drift away from that frustration and angst about as much as they would from a perception of inactivity... I think so long as you guys stayed active on the site, maybe making more frequent but smaller news posts about whats going on (maybe, "Hey, we just sent some ship balance changes to the testers!" or "new missions are in testing" - hell, even farm that off to a trusted tester), you would end up with a more professional feel to the whole operation as well as far less frustration.

I mean right now Waylon is basically getting tester feedback from people about his trade guild missions, as people uncover bugs and report potential problems/exploits. Snag spellcast and flog him into a test server - the guy already does it for you, and this way everyone else wouldn't have to go through the same grind of realizing somethings broke and waiting for the next fix to come out.

I'm sure you'll figure it out. :) Time for me to go restart my subscription... it must have died in the past few weeks.