Someone's Guide to Beta Testing - the golden rules

[Kroy]

If there's one thing that always happens in a beta test, it's people's expectations getting shot down. Beta test is a lot of things to different people, but one thing it isn't is a finished game.

The following text has done the rounds on a number of beta and pre-beta forums elsewhere, and I post it here for those new to beta testing who may not have read it before. I hope you find it helpful. I am NOT the original author of the remainder of this post, but I have edited bits here and there:


[Guide by another] Beta Testing - The Golden Rules

This guide is designed as a reference and reminder to all of us (and especially to those of us here who are relatively new to Beta Testing) of some of the fundamental issues and behaviours to consider whilst participating in this Closed Beta for Stargate Worlds.

I've been in lots of beta tests. This doesn't make me any smarter than anyone else here, nor does it necessarily make me any more valuable to CME, or indeed the Stargate Worlds project itself..
..but what it does give me is experience. This experience means I am able to realistically adjust my expectations when participating in any beta.

Why am I waffling on about this?
Good question!

I most certainly do not wish to come across as egotistic or condescending, so please bear that in mind. Instead, I would like to share with you a couple of little 'rules' I have developed over the years which help remind me of my place in a beta, and of what is expected of me.

I sincerely hope they help others reading this as well. =)


The first three basic rules of Beta Testing are:

1. Patience
2. Patience
3. Patience

Nice and simple, yes, but ever so difficult to truly adhere to.

What does it mean?
It means that in order to be a useful, valuable, and long-term beta tester, you have to steel yourself with oodles of patience. It's no easy task, especially in an environment such as this whereby the game mechanic is addictive and progressive in and of itself, but it's a vital component in terms of remembering why we're here and what it is we're here to do.

A couple of recent examples here on the forums have shown us that indeed patience might be a great virtue.

Really try to remember those three rules and set your basic expectations up to consider patience in everything you go to do here. It's a great little 'checkpoint' mantra to recite when things don't transpire as expected.


The second thing to always remember, is that in fact we are here to beta test a new application.

We are not here simply to 'play the game'.

This is an Important Point:
Directing our efforts towards testing, is essentially our prime directive here. 'Playing' is a component of testing. NOT the other way around.
Can't stress that enough (so I'll say it again)

'Playing' Stargate Worlds here, is a component of testing, not the other way around.

By 'playing' of course, I mean becoming emotionally invested in our characters, spending a great deal of effort to (for example) complete every Q at this early stage, and generally assuming that we are paying customers and that the character is 'ours'.

This is a fundamental (but highly understandable) mistake to make whilst you're here.

1. As has been mentioned before by developers of many in-beta games, you have to expect that your characters will be completely wiped - never to return - at any stage.
2. You also have to expect that the world will not be available 24/7, and that it may be unstable at various and multiple times.
3. You simply must must must expect to 'lose' everything you have invested your time into, at any point.


These three items are harsh in practice, but of course in reality they are par for the course.

When we sign up, we do so agreeing to be testers.
We did not sign up to progress a character flawlessly through to level cap, and nor did we necessarily sign up to give ourselves an entertaining RPG alternative-life.
(This is reserved for the release of the game when we, like all other customers, will be able to 'start' our characters for real.)

Effectively, we are volunteers with responsibilities. As such, we have a job to do - not a toon to nurture and raise.

Before you say "Hey Writer! Wait up - quests need to be tested through to completion in case there's a bug!?"
Yes, absolutely - but if you think about it, we are not even yet at the very beginning of the testing phase. Whilst it's important to run the quests, and especially the early zone ones which are easily accomplished, it likely makes more sense to only 'dip' into the later ones, until later in the Beta Program.

Why?
Because the further along the program we travel, presumably the fewer instances of total reset will occur. (I said 'fewer', not 'none' btw..)
You're likely better off spending your time getting through to higher zones, and experiencing a wider variety of content, than focusing on details such as this. As the patches come together, and the server becomes more stable (and therefore less likely to be completely reset) you will have more allowance in terms of providing value to the devs via grinding a toon through to completion.

Think of it this way: at an early stage such as this, it's best to dip into as much content as you can in order to find the 'low hanging fruit' bugs. A wider, shallower approach if you will. The time you spend on a lengthy Q is potentially better spent on progressing yourself to new content, allowing that new content to be tested.

As time goes by, there will be more opportunities to really delve into the details and catch the more buried issues.

Long story short:

If you are 'absolutely playing your character' as if you were paying for the privilege.. then you are setting yourself up for likely frustration and disappointment.

It really is that simple.

Instead - for sure dabble into all Q's, and run all the game has to offer until your eyes bleed, but hold ZERO expectations.

There are times whereby players should be respectful of the others here, but at the end of the day if there's an exploit, report it. Period. Once is discovery, twice is confirming, thrice is exploiting.

Report it and if the bug continues to annoy you then make a choice:

- stay in-beta, and just deal with it, let others exploit or glitch
or
- decide it's too annoying, you've done the right thing by advising the devs, so move along and wait for it to be fixed.

Additionally, don't hold any expectations (and especially, no 'demands'!) as to when the exploit or bug would/should be addressed.

The dev's are the only ones that can prioritise development, so lower your expectations and you may be pleasantly surprised by a fix sooner than you thought it would arrive.

Also, don't assume that just because there has been no 'formal' response to the issue, that it is not being worked on by the devs.

If it's been reported, it's in the system. It's most certainly not being ignored.
It may be prioritised down the list of things to do, but it hasn't gone unnoticed.


Finally - one extremely important thing to remember: We're all on the same side in beta =)

This comes back to my point earlier - of being too invested in the game. If you find an exploit, /report it as a first priority, then move on.
(If you are too emotionally attached, this can be difficult to remember, as you may feel like you are being 'cheated' out of play value. Remember once again: you are not a paying customer, and the toon is not 'yours' - you are testing.)

By all means raise the issue in the beta forum, however there is one more golden rule:

Critique the gameplay mechanic flaw, and not the player.
(obviously this need not apply if there are incidences of racism, abusive behavior, etc.. different kettle of fish there.. )

Whilst each and every tester should, in theory, be reading the beta forums, in reality this isn't going to happen. Don't assume that just because an issue has been highlighted on the beta forums, a player in the game will know about it.

What should you do? Either walk away or - if they are on your team, politely let them know it's a known issue, and ask them to stop

The best thing you can do, if it's unable to be stopped, is to move on to testing something else.

In the heat of battle, where our egos and our pride risk being dented, it's understandable how difficult this can be.

But that's why we always need to remember those first few rules:

* lower your expectations
* remind yourself that it's not 'your' character at this early stage
* and remember that we all are effectively volunteer co-workers of the dev team

With all that in mind, you can't go wrong!


I apologise for the length of this, and as I mentioned first-up I HOPE this doesn't come across as condescending, or make me look like a right prat, because thats most certainly not the intention.

It's highly possible that there are folk here who are new to Beta Testing. The more help and support we can give them as a community (in terms of what to expect and how to be most useful), the better for all of us concerned, and most importantly - the better for the developers themselves.

/guide]

[Kahless]

http://forums.stargateworlds.com/showthread.php?t=10980

[Haratu]

Many people try to get into betas to boast that they played the game, which is not true as several points are important:

- The beta is in place so that the game will change
- The end result may be better, or worse, than the beta
- The beta comes with work (reporting)

[Furau]

A very useful thread, in my view worthy of a sticky.

[Trollbasher]

Lol i was about to post something like this

The one thing i hate about alpha and beta testing is all the immature frealoaders who are only after a free game and bragging rights, all you here of them in the forums is complaints about testing going to slow, not enough patches, complaints about bugs not being fixed, they have no real idea about what is involved in game testing and development.

Great thread please sticky .

[steve30x]

Beta testing is a tedious job but it can also be fun. Just because you are in the BETA doesnt mean its going to be a whole lot of fun all the time. While you will be told what needs testing you will need to test that and then after the testing is done for a short while you can have fun. But while you play you may find a bug or two which you should inform your superiors.

[Tdunk]

Beta testing can be very annoying. If you want to play the game honestly wait till it comes out. Here's why.

-Disconnections
-Constant Character Wipes
-Client Issues
-Hardware Issues
-Memory Leaks
-Bugs
-Glitches
-Ect.....

This is just a chance for you to help out the developers and generally in return they give you like one month free of service from them, because you helped. Not sure what Cheyenne's policy is on that, but really if you want to do the beta just so you get a chance to play the game, dont. You wont even get the full experience that the developers intended.

[janete]

Good thread guys!


Before anyone gets any different ideas, it is good to clear the air. Well done. I certainly can't wait to be challenged and challenge and try it out. I have been playing MMOs for a few years, but as an SG fan I am very excited!

[Mighty.Chip]

Phat!!!

Good work, folks!

A good tool for while you're in Beta is to write down and rememebr how to take a screen shot of a single window. This isn't to take screen shots of the game, but to take screen shots of any error messages that pop up when the client completely bails out on you. Alt + Print-Screen will do the job. This will put only the window that currently has focus into the clip board. It's a lot easier to deal with that, especially if you're sending it back to the folks at CME. A screen shot of a single message box/error mindow takes up much less space.

Heck, if you're using a proper email client at home, like Outlook, Outlook Express, Mozilla Thunderbird... then you can just past the contents of your clipboard back into an email to fire off to CME, if they want to see the message box window.

Don't forget, once that's in the clipboard, you can just paste it into Paint... or MS Word... or Photoshop... etc.

[raizito]

Nice guide...
Well, I guess people must have in mind that bugs and exploits will be fixed in time. Could be during beta testing or after the game is released!!

The thing about beta testing is that you can help the devs fix the problem before the release, so the real custumers (yourself included), will have less patches with fixes and more with game content.

after all, they won't stop receiving bug/exploits reports after the game is released!!

[Brutaliss]

Originally Posted by Tdunk

Beta testing can be very annoying. If you want to play the game honestly wait till it comes out. Here's why.

-Disconnections
-Constant Character Wipes
-Client Issues
-Hardware Issues
-Memory Leaks
-Bugs
-Glitches
-Ect.....

This is just a chance for you to help out the developers and generally in return they give you like one month free of service from them, because you helped. Not sure what Cheyenne's policy is on that, but really if you want to do the beta just so you get a chance to play the game, dont. You wont even get the full experience that the developers intended.

well put, dont think if you get in the beta, its to get fun out of it, I've already seen beta tester, quit beta testing because of character wipes, and they wanted to play to have fun, not to find bugs and make the game a better game.

Beleive me, theres nothing more enhancive than finding a bug or an exploit being able to recrated the said bug/exploit and then sending a report to the Dev. and when the next patch comes out and the problem has been fixed, you feel like you did your part to make a better game.

[Merlyn]

I always think that beta testing is a little like being a detective....

[Diiddie]

Nice!

[Trollbasher]

Originally Posted by Merlyn

I always think that beta testing is a little like being a detective....

Thats what i have always liked about testing games the searching and finding the bugs just like searching for and finding the suspects .

[Hobbitgoth]

I'll probably get flamed for this, but you're being an ***. Sure you have good points, but don't be so harsh about them Kroy. I agree with your subject, but not your delivery. And yes, I know you're not trying to be harsh.

[TripleG]

anyone here play the beta for navy field? *shudders* man that sucked, the game balance was terrible, and torpedos exploded like they were nuclear weapons, hitting multiple ships. worst of all the players were mindlessy shouting OMFG NERF NERF NERF! at every major inconsistancy. And when the developers actually listened to the players, the results were terrible!!

So my advice would be, if you think somethings unbalanced, Be calm and polite about it. Please think before you post, and try to come up with a good, or creative solution, preferbly one thats both.

[Trollbasher]

What gets me is those people who complain about things in the forums bugs and everything but never actually make a bug report about them.

[Ariot]

"Remember once again: you are not a paying customer, and the toon is not 'yours' - you are testing."

Extremely important, there's no use whining over and over again about the same problem that's already been reported or after getting wiped, just because it's annoying, you just have to be patient and continue to explore.

[Trollbasher]

Originally Posted by Ariot

-- A few pints of beer is what makes most men cheer --

Sorry onto my 7th pint "500 mil can 3 1/2 L" what am i cheering about

And yes i fully agree i wish there was some form of test that you would have to pass before you could test games or for that matter even play them .

[Anteaus]

I concur, Kroy.

If there's anything worth getting invested in during Beta, it's the game itself - "play" it such that you're doing everything you can to make it as good as it can be.

Wait to inveset in your character until the game is released.

[masterpunks]

Thanks kinda reminds me of when I beta-tested Lord of the Rings online, which was working quite well for the stage it was in, but there some pretty good glitches in the crafting system. A few quests didn't work out quite like the should but I didn't mind cause you have to realize that the game isn't complete, you check the forums and see if the team knows about the glitch and then report it if they don't, and move on. Sometimes I got kicked off randomly, and well that is just part of the deal.

[nekosan26]

Some very good rules to live by when beta testing, i've beta tested a few mmos myself, and you cant expect everything to function properly till they have been tested and tested again. Thats why they do beta testing, then we get to see some positive stuff in the Live version of the game and its because as a beta tester you helped fix it

[DannySGC]

Nice post there.

The three basic rules. Is a must !

Fellow these and you wil go far and might be a GM in any game. If you help out etc and so forth.

[DannySGC]

Originally Posted by Trollbasher

Lol i was about to post something like this

The one thing i hate about alpha and beta testing is all the immature frealoaders who are only after a free game and bragging rights, all you here of them in the forums is complaints about testing going to slow, not enough patches, complaints about bugs not being fixed, they have no real idea about what is involved in game testing and development.

Great thread please sticky .

Dam bloody right ! I agree with you on this one !

It takes people time up to get the code right.

When you have like 1000+ lines of codes. It really dont help when you got some little to faced. Going on and on and on etc.

[Ames]

Valuable info for beta testers and all true! You are there to find problems, which may mean dropping what you are doing and moving to a requested task. You may have to do that task repetitively on occasion. It's fun, don't get me wrong , but it's a different kind of fun than just playing the way you might want.