Odinseye's Mini Beta Testers Guide

[Odinseye]

In an effort to help any here that may be new to the Beta testing process I have decided to re-post this guide that I wrote that has helped many others in various different communities drawing near to a testing phase of game development.

Odinseye's mini guide to better beta testing

With the upcoming alpha/beta test of Stargate Worlds (or "SGW" as I will refer to it in the rest of this post) on the horizon, I thought I would post this small guide I wrote. In an attempt to help people who may be new to the process of beta testing be more effective, and efficient. Also to help them understand the reason and the function of a testing phase of a games development.

To help present what may be expected during the testing phase, so that an idea of what lies ahead is clear. Also, to help people understand that getting accepted into alpha/beta, is not a status symbol, or a show off point. It is a tool that developers employ to hone their product. It is not a free preview of the game to "try it before you buy it". Although one can get a good idea of what the finished product will be like, but if that is your main focus then you have missed the point of it all. The main focus of a testing phase is to make the best product possible, period.

For all those that haven't alpha/beta tested a game before, what should you expect?

Well, honestly that depends on what stage of alpha/beta the game is in currently when you join in. If the game is still in its Alpha stage, all manner of chaos may be present. Disconnects, lock-ups, crashes, lag, character wipes, frequent patching, etc. Usually the alpha stage of a game development is initially for the developers and staff or family of the gaming company, and a select few others. It is usually, but not always, characterized by unstable situations where crashes are prevalent. In this stage, (unless massive amounts of play testing have been done) most games and game systems, are not too dependable.

After some quality testing and bug reporting, most major issues usually are handled quickly. The length of the alpha stage varies by game company, and their feeling of readiness to move to the next stage. Some company's may have a few months of alpha, some may last over a year, or longer.

If a game is in its beta stage when you join in, then some of the major issues may have been handled, but always expect the unexpected when humans are involved. There is nothing better than a squad of rabid, ########, gamers to break game systems.

The beta stage is usually where the majority of testers are allowed access to the client. Populations during beta testing can get high, but that depends on the client and server stability. Usually in the later stages of testing, companies will release the floodgates and allow lots of testers to join to test server strength and stability, this phase is usually referred to as "open beta". For a company to run simulations and to play test is one thing, but having a few thousand gamers on your servers is quite another.

So while you are in testing phase, don't expect the game to be fully functional and 100% playable. Don’t get caught up in trying to advance your character, and don't get too attached to your toon, cause the next time you log in it may be gone.

Expect to agree to a "NDA" or non disclosure agreement, you will not be allowed to enter testing without this consent. Basically what it will boil down to is that you are going to be given trust by the company producing the game to take part in their software and that you will not leak or reveal their secrets about their game to anyone. You are given a very great trust, as the info you will be seeing will be top secret, do not betray this trust! You will also be required to keep the URL of the reporting forum secret, posting content from this forum to anywhere on the web is a big no-no. Depending on the company and their stance on their software, leaking info could land you in court as well, so keep this in mind when you are tempted to "show off" to your homies on the command website, forums or wherever.

Where do I report the bugs or issues I find?

Most gaming company's, in the event of a alpha/beta test will set up a forums at a secret location with only tester members allowed access to report bugs, discuss issues the game is having, and to leave your feedback. Communication is crucial to the game developers in getting problems found, then corrected. Most reporting forums will be broken down into different sub-forums for different game issues. Combat, toon issues, travel, skill systems, chat, etc.

All manner of issues will be present. Be accurate in your posting of reports, make sure they go in the section most related to the issue you have found. Also scan the posts on the forum to make sure that you are not double posting the same bug someone else did the day before. Getting all the info related to one particular bug in the same thread can save the developers lots of precious time and searching.

Some of the newest games lately have had features like screen shot tools or in game report functions to make reporting easier and more clear, these are nice but not all do this. A lot of games will also have auto reporting features that will send info in for you, sometimes without you even being aware you just had an issue!

What kinds of things should I include in the report or post?

Be clear, concise, and explain your bugs to the best of your ability. Always include key issues to the finding and reproducing of bugs, being able to reproduce the bug is also crucial. Can you repeat it? Did you do it the same way each time or is there multiple ways to create the bug? Are there ways to work around the bug? State those as well, as they may be helpful in determining a solution to the problem.

Things like coordinates, XYZ if present. Location is key in some issues as some bugs will happen in specific locations like instances and nowhere else. Include things like what you were doing when you noticed the issue. What was your toon doing at the time? If you were trying out a game feature like a skill or a ability etc, then state that as well. Report what the npc's or mobs were doing, and which were present, as sometimes specific mobs or npc's actions may be the cause of the bug. Report the actions of other player characters in the area also, as other players grouped in an area, or their combined actions, or numbers in the area may be the cause of the bug. Basically take note of everything, your surroundings, your actions, your inventory, other peoples actions, presence, etc. Be sure to fill in all the required info on your bug submittal form as it is there for a reason.

What things can I do to produce bugs or other issues?

Basically the best answer is just try it all! Play the game, try the systems, journey to the locations, work with the npc's, equip or use inventory. A bug or a glitch can hide anywhere, in any game system, waiting for you to find them and report them. One thing that has been invaluable to me in the past in finding bugs is to use the info already presented in the reporting forums. A lot of times one can find bugs by doing similar things others have already done when they found the bug they are reporting. Some particular game systems might be plagued with numerous bugs and similar issues.

Lots of things can be used to find bugs, lag for one is notorious for creating bugs. Try using or doing things in areas that are experiencing lag. Try out all the games systems, not just the combat. Keep an eye on your statistics as some items and or equip-ables may alter your stats in a way not intended by the developers. Keep a close eye on the damage you take while in combat as that can be used as an indicator that something is amiss if by some reason you are no longer taking damage, or the other way around and you are taking too much.

In the end what are my responsibilities?

If you are selected to help test SGW then do us all a favor and repay that honor with some basic tester responsibilities.

1. Do your part. When you find a bug or any kind of glitch, take the time to report it, and do it neatly and accurately. Even if you only find one bug every other week, you have still helped make SGW a more bug free environment for the rest of us.

2. Keep the trust you have been given. Don’t betray CME and the faith they have placed in you to test their product. Do not release any information about the game at all, until the NDA is lifted and you are free to do so.

3. Be courteous of all your fellow testers in the realm as well. This one is pretty self explanatory for most mature gamers.

If you will do these 3 things you will go along way to the improvement of SGW. A game made more free of bugs, and a general bettering of the game realm by your involvement.

I truly hope that this information will help you in your quest to become a better beta tester. The information I have written here comes from my past experiences in testing environments, the most notable being beta testing UO, EQ, Neocron, DDO, LOTRO, Vanguard.

~Odinseye.

[lermanb]

Nice job Odinseye. Strong work.

[Rae'f]

I think this deserves a sticky. And it'll hopefully help those new to beta's and/or those that misunderstand the point of betas, to help make SGW a better bug free world.

[Odinseye]

Thank you very much, I worked hard on this and I have only found it plagiarized on the net three times so far, lol.

[Leadfingers]

Originally Posted by Odinseye

In the end what are my responsibilities?

I would add this:

> Understand that your vision for the game is not CME's vision for the game. You have to be able to see the game from thier point of view, and without being burdened by your own opinions or you aren't much good to them.

> Understand that just because you've played every game since The Realm Online, that doesn't make you knowledgeable, or a good tester. I've been married for a long time. That doesn't make me great at marriage.

> Leave the drama at the door.

> Leave your 1337'ness at the door.

> Understand that anecdotal evidence is crap. If you can't measure it, repeat it, and document it, then it didn't really happen.

> Respect CME. This doesn't mean that you have to be a "yes man". Lord knows I've fought some hard fights with game companies. But understand that you are an invited guest into an area that they work in 40 (or more) hours a week. Try to act like you deserve the courtesy that you've been given.

> If you don't know something, try not to act like you do. I've seen testers go on for HOURS (literally) talking around and around circles, arguing about data that none of them had any real knowedge about. It was amazing.

> Learn to love the phrase, working as intended. Just because you don't like a skill, that doesn't mean it isn't working as intended. If you want to change it, you need to build a slow, strong case to do so.

> Be patient. Your schedule isn't thier schedule. I've bugged things for almost a year before they got implemented.

> Your feedback is important...far beyond simple bug reporting. Bug reporting is the easy part. Balancing, thinking, evaluating, analyzing... seeing problems from multiple angles and proposing compromise solutions...The testing community serves as a sounding board for the development teams.

[Odinseye]

Hey thanks for the good discussion on this.

By all means if anyone has some good advice from an experienced point of view to add, be my guest. If it all was in an easy to access thread all in one place who knows how many people it might just help you know?

In the end the better job we do as testers, the better the product we will have to call home after release.

[Galahad]

Thankyou this has really helped me understand more of what happens in a beta. Thanks

[Odinseye]

you are welcome for sure. I am just really glad to be able to help someone out in some way.

[Menzel]

This should totaly be stickied. I can't tell you how helpful it would have been to have a post like this the first time I was in a beta test.

Its REAL important to take an objective view of the game you are testing. Don't let your opinion sway what might work just fine just because you don't like it. On the other hand don't feel that you shouldn't make an effort to get something changed if you really feel its just not right, just don't get upset if others and/or the developers don't agree with your view.

[tommythecat]

Very solid points here, I've been beta testing games for years and it would've been nice to have a guide like this when i first started out doing it. It's become second nature when i play MMO's I play as if I'm in beta trying to find bugs and glitches.

[Odinseye]

@Menzel, I totally aggree with you on your point, it is very important to remain objective when testing a game, being biased to a degree that you can't do your job only skews your efficiency for sure.

Originally Posted by tommythecat

Very solid points here, I've been beta testing games for years and it would've been nice to have a guide like this when i first started out doing it. It's become second nature when i play MMO's I play as if I'm in beta trying to find bugs and glitches.

Lol, I totally do that as well, it's funny how seriously testing some games can make you alter you playstyle like that in future MMO's. I am always on the hunt for bugs and such when I play the ones I play now.

I find a certain kind of satisfaction on finding errors and then reporting them to get them corrected. There has been so many times in my many years of playing MMO's that bugs and such have really hurt my experience.

Down with the bugs!

[Gizza]

Thank for the post odinseye mate.

Ive always wondered what goes down in a closed beta test phase.

Ive probably just shot my self in the foot for posting that im not an experienced Tester but i take comfort in that hopefully someone who knows what their doing will be able to help CME better then i could.

Hopefully the more experienced testers get chosen over those who want to only see and play the game.

[DrFunk]

I just like finding ways to sneak into areas you shouldn't be able to access. Such as the top of houses in a city. Seriously though, it's a good guide. I'm sick of beta testing a game only to find out that it's hard to report a bug because the forums are full of people whining about things that work as intended and are never going to get changed anyway.

[dragon_sword]

Originally Posted by Odinseye

Thank you very much, I worked hard on this and I have only found it plagiarized on the net three times so far, lol.

Hi i was whondering do b com a beta or gameing tester
some online gamers allways till me
i shod b a beta or gameing tester

for expole on Call of Duty 2, me and 2 nubs go agints
13 players and i use the springfield or gumand snirper rifle

[Menzel]

Ok dragon_sword, don't take this the wrong way but you may want to seriously work on your communications skills first.

While I'm not one to point fingers since a lot of my grammer and spelling tends to suck, the developers tend to get the most out of well written, thought out responses that they don't have to decipher.

(I just know I've been suckered into taking someones troll bait....)

[longbow]

I don't think too many people here will have to worry about getting into an alpha test. Those are reserved for the dev team, and its QC people, plus MAYBE a few "friends and family" of those people. Alpha is definitely not something you should subject random people to. And even early beta stages, it'll probably remain "friends and family" in size, to keep the burnout low, and the quality high. The biggest danger to a MMORPG like this is inviting a few thousand people into your beta, only to find out you have to rework the entire thing to (try to) make it fun. Warhammer closed their beta for a few months because of this, which is probably a wise decision instead of letting their beta testers become bored testing obsolete content/mechanics.

Once "Open" beta (AoC's version of open is "not quite as closed" beta) comes around, you pretty much CAN use it as a "trial", as the bugs should all have been reported, and the polish has been or is being applied to the last little bits. Major changes to gameplay are contraindicated at this stage of beta. The "open" beta servers two purposes nowadays: Marketing and hype (if your game is "good and fun", this will work greatly to your benefit: LOTRO and WoW are good examples) and server stress testing. While there are tools out there to simulate stressing your systems, nothing beats having 10,000 rabid players trying to play your game all at once to find those bugs you'd never find otherwise in your server code (lag induced dupe bugs comes to mind from past experience, or the opposite, vanishing item bugs). If your game is poopy or not fun or broken, the open beta or large closed beta will work against the developer (Tabula Rasa comes to mind).

So there is a phase of "beta" that isn't about finding bugs .

[bctrainers]

Looks good, however i am pretty sure their NDA might be a little more stricter than the ordinary NDA's out there today.

[TheAincient]

Everything stated here is fairly standard testing procedures.

I disagree with one thing stated, and thats about reproducibility.
If during testing you encounter a serious bug or issue but you are unable to reproduce it it should still be reported.

These types intermittent bugs are **** for developers (I know since I'm a software engineer) but the devs still need to know that something has happened even if they cant do anything about it, usually though they will have an idea about which module the problem occurred and can put in defensive code.

[Lemon]

Thanks for taking the time to post that thread, covers then basics for people who have never beta tested before and gives them some simple and easy guide lines to follow.

[ignite]

Great guide, thanks for taking the time to post it.

[magecu]

Would you want me to merge this thread with Khestrals Beta tools thread?

If so just PM me.

[Amenopophis]

Also thanks from my side, I also have never beta tested a game. That's about how I would have done it anyway I think, but that might come from the fact that I've studied electrical engineering and also had some programming lectures (C/C++, Java and Matlab) and how difficult it can be to find bugs. The thing about testing the different system parts within a laggy area is a good point that I didn't think about at all.

[VVarlock]

Thanks for the post, I have never gotten into a game early enough to help with beta, but the guide seems straightforward enough.

One thing I always hated in game was the super l33t peeps who had all attitude but little input on how to make things better.

Warlock

[blkjag]

I would also add the following:

1. Post your bugs in a positive and constructive manner

2. Show respect to your other beta testers and especially to the developers in your interactions and in your posting to the beta forum. Inappropriate languages and abbreviated curse words are not allowed.

3. Avoid using open chats to find out or get information on a bug or saying statement "have anyone experience this before? "etc" please read the beta forum for the latest update. Use the pm if there another beta tester around and ask them to help you to duplicate it and see if they can- this will help to determine where the bug might be, on your system or in the program itself.

4. Follow the schedule and program that the developers layout. They may want you to test a certain functions for 'x' period of times for 'x' numbers of day, its may not be something you like to do, but it is necessary.

[Odinseye]

Wow there has really been alot of positive ideas and some really solid thinking that has come from the responses to this thread, thanks. That alone has made the effort in writing this worth it.

Originally Posted by TheAincient

Everything stated here is fairly standard testing procedures.

I disagree with one thing stated, and thats about reproducibility.
If during testing you encounter a serious bug or issue but you are unable to reproduce it it should still be reported.

These types intermittent bugs are **** for developers (I know since I'm a software engineer) but the devs still need to know that something has happened even if they cant do anything about it, usually though they will have an idea about which module the problem occurred and can put in defensive code.

I agree with you in this post you made 100% I may not have been clear in the way I presented the idea on that thought, sorry. I agree that all bugs should be reported no matter how minor you may believe them to be or if you can't find a way to reproduce them.

I just wanted to convey the ideas that bug testing should be done with an attitude of alertness so that you can try to reproduce if possible the steps or happenings that led to the event. That makes a developers or QA or coders job so much easier when they don't have to dissect an issue to determine its cause.

If they have an issue and its cause reported together, then it becomes much more efficient to begin the process of fixing it and then moving on to the next issue, as they will no doubt have a multitude to deal with.

I agree with you for sure though, all bugs need to be reported, reproducible or not.

@Blkjag Some good thoughts that you had as well. I especially agree with #4, I didn't think to mention that and it's a very good point. In past beta experiences we have seen many times that the developers do have a certain area or mechanic that they want worked on.

I would say to all serious testers out there to make sure and keep up to date with the beta forums when they arrive, so that you can see when the developers are asking for tester help in certain areas. To help them focus and maximize their efforts when they need to.

Great point!