Stargate Worlds Game Site | Game FAQ | Community Forums | Command Forums | Dev Post Tracker

Go Back   Stargate Worlds Game Forums > Stargate Worlds MMO Discussion > Beta Discussions
Register FAQ Members List Calendar Search Today's Posts Mark Forums Read

Post New Thread Reply
 
Thread Tools Display Modes
Old 12-10-2007, 02:42 PM   #1
Khestral
Firesky Staff
Cheyenne Staff
 
Khestral's Avatar
 
Join Date: Mar 2007
Default Beta tools, give your ideas.

Quote:
Originally Posted by SnakeCharmer View Post
Please please please please have a powerful, concise bug reporting tool available during beta.

Please please please please have simple quest surveys during beta.
Ok so here is something I can work with

We talk about this stuff a lot at work and I want your thoughts on it now... What do YOU think would work for this type of thing?

Tell me what your thoughts are on this
__________________
Firesky - Stargate Worlds - Stargate Resistance

Director of Quality Assurance

Last edited by Scepter : 02-25-2008 at 09:44 AM.
Khestral is offline   Reply With Quote
Old 12-10-2007, 03:07 PM   #2
SnakeCharmer
Banned
 
Join Date: Feb 2006
Location: The Castle
Default

Quote:
Originally Posted by Khestral View Post
Ok so here is something I can work with

We talk about this stuff alot at work and I want your thoughts on it now... What do YOU think would work for this type of thing?

Tell me what your thoughts are on this
Heh.

Funny you should ask. I started typing something up not too long after I made the post. Be warned though, it's very, very rough, and not knowing details about the game and how it works, I could only make something that in of itself is very broad and general:

Bug reporting tool (rough draft)
1) Character name automatically put in
2) World waypoint: automatically put in, or easily obtainable by the player via UI
3) Is this quest related? Yes / No. (If yes, then drop down to name of quest from characters quest log NOT every quest in the game). If no, then skip to 4.

3a) Drop down listing name of quests in user journal
3b) Is the bug repeatable? Drop down Yes/No
3c) Description (with character limit)
3d) Submit, with autogenerated response / confirmation of submission email (ingame as well as out of game) to the player with a confirmation number for reference.
4a) Nature of bug/incident. Drop down menu: Character/avatar issues, World Geometry, Abilities/specials, World interaction (switch not working, npc unresponsive, etc), Exploit (dupes, etc), GUI issues, etc. Each drop down could/should have subtopics for further refinement.

The goal here is to narrow down the issue to determine which team it should go to for resolve. Ideally, all teams should be divided up (art, world builders, etc even if only on a broadcast email group) and bug reports parsed to the relevant dev teams if at all possible, or alternatively (and probably the best avenue), all bug reports should go first to QA for immediate follow up and replication, then sent to dev teams responsible for that area. QA should flag each report with a score indicating severity and urgency. The two are not directly related; a bug may be severe, but not urgent, and vice versa (example: a dupe exploit may be simple to replicate/fix, but its urgency is paramount). If QA is unable to replicate, then someone from QA should contact the person who submitted the bug for further follow up.

4b) Description (with character limit) - You don't want too much to where the info becomes convoluted and useless, but you don't want too little so that you can't get an accurate description.
5) Submit, with autogenerated response / confirmation of submission email (ingame as well as out of game) to the player with a confirmation number for reference.

I would argue that the drop down menus need to be as thorough as possible to help YOU guys isolate the issue as quick as possible. The problem for you guys is that players WANT to submit a bug, but at times it can be overwhelming trying to accurately describe what is happening. In beta invites, I'd stress the importance of a pen/paper to keep notes, or alternatively have a /notepad command in game that players can use in leiu of (it'd be nice if this could somehow be attached to the bug report...).



Quest Surveys:
Pretty easily done. POTBS had a pretty nice one although at times it became a nuissence. The biggest problem with surveys is you depend TOTALLY on your players to be honest and unbiased, both to get an accurate answer as well as us not just answering <whatever> to get it out of the way. Too many surveys or too many questions and we tend to have our eyes gloss over and click random numbers and get it over with.

The magic number of questions for <me> is three...
1) Did you find the quest entertaining? On a scale of 1 to 5 (with 5 being very entertaining), how would rate this quest?
2) Did you find the quest challenging? On a scale of 1 to 5 (with 5 being very challenging), how would rate this quest?
3) Did you find the reward for the quest be worth the risk and/or time involved? On a scale of 1 to 5 (with 5 being 'worth the risk/time'), how would rate this quest?


It would do wonders for you guys to have a background process that snapshotted how many people were in the group, what levels they were, and what classes they were, as well as the level/class the person filling out the survey was. It would give you a reason as to why someone thought a quest was too easy (too high of level versus the intended level of the quest, was it an intentional solo quest that the player did with a full group? Did the player try a quest that was intentionally set for a group/higher level? This is, of course, dependent on whether or not NPCs will give out quests to players outside the level range of the quest line the NPC is giving...)

You could possibly have an optional 'Comments' section where players can freely put thoughts about the quest down, with character limit. They don't have to fill out the comments to submit your survey answer, to avoid getting stuff like adkjfhakjhsdfkhaskdf and 'submit'.



Probably more than what you wanted, but there you go...
SnakeCharmer is offline   Reply With Quote
Old 12-10-2007, 03:43 PM   #3
blkjag
 
blkjag's Avatar
 
Join Date: May 2007
Location: San Francisco Bay Area, CA
Default

Quote:
Originally Posted by Khestral View Post
Ok so here is something I can work with

We talk about this stuff alot at work and I want your thoughts on it now... What do YOU think would work for this type of thing?

Tell me what your thoughts are on this
Quote:
Originally Posted by SnakeCharmer View Post
Heh.

Funny you should ask. I started typing something up not too long after I made the post. Be warned though, it's very, very rough, and not knowing details about the game and how it works, I could only make something that in of itself is very broad and general:

Bug reporting tool (rough draft)
1) Character name automatically put in
2) World waypoint: automatically put in, or easily obtainable by the player via UI
3) Is this quest related? Yes / No. (If yes, then drop down to name of quest from characters quest log NOT every quest in the game). If no, then skip to 4.

3a) Drop down listing name of quests in user journal
3b) Is the bug repeatable? Drop down Yes/No
3c) Description (with character limit)
3d) Submit, with autogenerated response / confirmation of submission email (ingame as well as out of game) to the player with a confirmation number for reference.
4a) Nature of bug/incident. Drop down menu: Character/avatar issues, World Geometry, Abilities/specials, World interaction (switch not working, npc unresponsive, etc), Exploit (dupes, etc), GUI issues, etc. Each drop down could/should have subtopics for further refinement.

The goal here is to narrow down the issue to determine which team it should go to for resolve. Ideally, all teams should be divided up (art, world builders, etc even if only on a broadcast email group) and bug reports parsed to the relevant dev teams if at all possible, or alternatively (and probably the best avenue), all bug reports should go first to QA for immediate follow up and replication, then sent to dev teams responsible for that area. QA should flag each report with a score indicating severity and urgency. The two are not directly related; a bug may be severe, but not urgent, and vice versa (example: a dupe exploit may be simple to replicate/fix, but its urgency is paramount). If QA is unable to replicate, then someone from QA should contact the person who submitted the bug for further follow up.

4b) Description (with character limit) - You don't want too much to where the info becomes convoluted and useless, but you don't want too little so that you can't get an accurate description.
5) Submit, with autogenerated response / confirmation of submission email (ingame as well as out of game) to the player with a confirmation number for reference.

I would argue that the drop down menus need to be as thorough as possible to help YOU guys isolate the issue as quick as possible. The problem for you guys is that players WANT to submit a bug, but at times it can be overwhelming trying to accurately describe what is happening. In beta invites, I'd stress the importance of a pen/paper to keep notes, or alternatively have a /notepad command in game that players can use in leiu of (it'd be nice if this could somehow be attached to the bug report...).



Quest Surveys:
Pretty easily done. POTBS had a pretty nice one although at times it became a nuissence. The biggest problem with surveys is you depend TOTALLY on your players to be honest and unbiased, both to get an accurate answer as well as us not just answering <whatever> to get it out of the way. Too many surveys or too many questions and we tend to have our eyes gloss over and click random numbers and get it over with.

The magic number of questions for <me> is three...
1) Did you find the quest entertaining? On a scale of 1 to 5 (with 5 being very entertaining), how would rate this quest?
2) Did you find the quest challenging? On a scale of 1 to 5 (with 5 being very challenging), how would rate this quest?
3) Did you find the reward for the quest be worth the risk and/or time involved? On a scale of 1 to 5 (with 5 being 'worth the risk/time'), how would rate this quest?

It would do wonders for you guys to have a background process that snapshotted how many people were in the group, what levels they were, and what classes they were, as well as the level/class the person filling out the survey was...

You could possibly have an optional 'Comments' section where players can freely put thoughts about the quest down, with character limit. They don't have to fill out the comments to submit your survey answer, to avoid getting stuff like adkjfhakjhsdfkhaskdf and 'submit'.


Probably more than what you wanted, but there you go...
Adding to this particular topic -
A standardized template and objectives that everyone must follows - have to be driven by CME

Drop down menu or list of the key features or categories to report the bug under- along with a numerical ranking system that is develop by CME that the beta tester can assign like (1-5) example where five is severe, and the bug is a kernel fault or system crash type.

A well thought out beta testing schedule of what key components/elements to test along with a Calendar that beta testers can access to see what components that the dev want the beta testers to focus on and test out on when they sign on that day and for how long (days or week). We here to help CME, but CME have to drive the process and standards and provide us the framework, tools and formats that will make it easy for the beta testers to report and for CME to track the bugs and maintain the data.
__________________
Encourage Positive Behavior and GamePlay
-
Dial the Gate, Let the Adventure Begin !!!
blkjag is offline   Reply With Quote
Old 12-10-2007, 04:51 PM   #4
Cythis
 
Cythis's Avatar
 
Join Date: Jul 2007
Default

If you go with drop downs just make sure to include one area where we can place a description of the issue. Lots of times the drop down doesnt suit the needs of the bug.

Also a beta working ingame bug knowledge base would be nice. This can also be including in the beta forums so we have access off line to update other issues associated with the bug. Alot of times I will /bug the same quest with multiple issues. This can also be used as a great tool to let us know to re-test fixed content.

Consistant beta events.. a little off topic, but run a different event every day or even 2 to 3 times a week (based off level content, quest series, etc..) .I did this in only one other mmo and they did it once or twice but it was so fun and a lot was accomplished.
Cythis is offline   Reply With Quote
Old 12-10-2007, 05:04 PM   #5
Mister B
 
Join Date: Dec 2007
Location: Minneapolis
Default

Continuing on this topic, it would be nice to have a /stuck command. Make this command do two things. First, it creates a log file that contains whatever information would be useful to you for finding the hole, like current coordinates and, if possible, pathing information for where the character was moving about for the past minute or so. The second thing the command does is teleport you to where you were physically one minute ago. This should get you unstuck without the potential of people just using the command as a quick teleport home.

I agree with SnakeCharmer, the more robust the bug report tool the better. Having it autopopulate with information like character race, class, level, location, current mission and team size is great. Also, make it clear that you are capturing that information, so folks don't feel the need to duplicate it.

Also, please put a small textbox for suggestions on the quest survey, or at least on some of them. And if you are going to have quest arcs, only put one survey at the end of the arc, instead of one popping up after each piece of the arc. I always have a better feeling for an overall arc than I do for one part of it.

Lastly, some manner of suggestion box. I would recommend a section of the forums that is specifically geared to gameplay, quest and UI suggestions. Make it known that the mods will have a heavy hand there, and things that are off topic, or generally not on point to the discussion at hand will be cleared out, so the powers that be can easily find and respond to suggestions.
__________________
The Old Timers Guild
"Laid back, not so serious, no drama.
All about the fun!"

www.oldtimersguild.com
>> Open to the Public! <<
Mister B is offline   Reply With Quote
Old 12-10-2007, 10:31 PM   #6
Raleighgamer
 
Raleighgamer's Avatar
 
Join Date: Oct 2006
Location: Fayetteville, AR, U.S.A.
Default

Quote:
Originally Posted by Khestral View Post
Ok so here is something I can work with

We talk about this stuff alot at work and I want your thoughts on it now... What do YOU think would work for this type of thing?

Tell me what your thoughts are on this
Simple quest surveys should ask some basic questions...

Name of Quest:
Did the story element of the quest grab you?:
Did the story element of the quest fit into the overall storyline of SGW? (could be tough to know that in beta):
Did the quest giver provide sufficient information?:
Did the quest clues provide enough direction for you to locate quest objective?:
Where the quest objectives logical (i.e. killing wolves has nothing to do with salvaging naquadah generators)?:
Was the quest fun?:
What was the best and worst feature of the quest?:
Enter additional comments:

The problem I've had with many games I've tested is that there's only room in the forms for short answers. I understand that devs don't necessarily have time to read a thousand novels written by testers, but by not taking the time to read all that the devs aren't getting complete answers to important questions. Please don't limit tester comments to short summaries, but instead let testers fully explain what they are trying to express in bug and content reports.
__________________
New to SGW? Find a summary of info on the game here

Raleighgamer is offline   Reply With Quote
Old 12-10-2007, 10:52 PM   #7
Liliane
 
Liliane's Avatar
 
Join Date: Jul 2007
Default

In my opinion forms aren't so good for feedback. I mean, problem is that not all feedback fits to pre-define form and selections. I have had previosly problems with that. If it fits to form then of course it's okey, but what if it doesn't?

I'm more for like free feedback person. Because sometimes, I report bug, when something isn't working, I can be agaist color used in somewhere, I just make suggestiong because something is missing or because I could not do some logical action, because quest or somethinng doesn't allow. I ques I'm developers nightmare as beta testers, because I'm not walking pre-define roads, I explore possibilities.

I do how ever agree with others that the feedback tool should be something what takes alot of pre-define information about player and situation where player is. Also it would be good if beta testers could fill the base information about they systems only ones. I'm not gonna tell everytime in reports what my computer cpu or OS is. Just store it in database one time.

I'm sure developers know what to ask from use, but remember that some of us, could do something unexpected. Like been outside the zone map.
__________________
Account: Liliane
Explorer 86.67%, Socializer 73.33%, Achiever 40.00%, Killer 0.00%
Read SGW FAQ and Wiki for more information.
Liliane is offline   Reply With Quote
Old 12-10-2007, 11:30 PM   #8
Jayna
 
Jayna's Avatar
 
Join Date: Apr 2006
Location: P3X-888
Default

I love forums for feedbacks.

SWG changed its entire death system because of the argument on a forum by first closed beta people

From a save state system where your clone if not updated you lose all that you gained to something less punishing.

It was forums not reporting, it was discussions open to all beta testers and QA people.

It wasnt opening a bug report and filling in a little form and then having that form go off to god knows where.
__________________
Just say NO to the Warcraft static storyline and YES to the Asheron's Call model where things change, end and progress.
Jayna is offline   Reply With Quote
Old 12-11-2007, 01:09 AM   #9
magecu
Moderator
 
magecu's Avatar
 
Join Date: Apr 2006
Location: Slovenia
CME Beta tools, give your ideas.

In another thread one of our devs, Khestral, has asked us to give our opinion about various reporting tools for when in beta.
In my opinion this is an enough important topic to deserve it's own thread.

So I have opened this one and copied all the relevant posts (if anyone is missing just PM me. ).

So here you have it, fresh from the gate.
Post away at your hearts content about the topic.

magecu
__________________

Don't limit your imagination!

Last edited by magecu : 12-11-2007 at 01:20 AM.
magecu is offline   Reply With Quote
Old 12-11-2007, 02:39 AM   #10
Amro
 
Amro's Avatar
 
Join Date: Nov 2006
Default

Be careful with forms- too many fields and it gets confusing. As others have suggested, as much information as possible should be automatically gathered (player stats, location, system specs, current settings, etc). Also, I think Yes/No should be implemented as option buttons rather than a dropdown box.
Amro is offline   Reply With Quote
Old 12-11-2007, 04:10 AM   #11
Ortrillian
 
Join Date: Nov 2007
Location: TX
Default

Definitely have an "other" option to report bugs. I've run into a lot of bugs that don't fit any category while testing other games. It's very frustrating to decide what category some bug will belong to with only a few options. Also, it makes it harder to get the bug info to the correct person if certain bug types are being handled by different people.

As a side note, PLEASE don't blow off weird bugs that only a few people report. I found a bug in the WoW mail system that kept getting blown off because no one else had reported it. It took almost a week of emailing different Devs to have someone look into the problem thoroughly. After finding the right person the bug was fixed.
Ortrillian is offline   Reply With Quote
Old 12-11-2007, 04:10 AM   #12
nightwalker
 
nightwalker's Avatar
 
Join Date: Aug 2006
Default

Forums all the way!...
While beta testing a few games i reported numerous amounts of bugs via the in-game tool...at release around 80% of those bugged remained ingame,they all varied from broken quests to npc's floating around in mid-air.
I'd much prefer a forum site where we could actually get dev confirmation that they are looking at the bugs...
nightwalker is offline   Reply With Quote
Old 12-11-2007, 04:13 AM   #13
Amenopophis
 
Amenopophis's Avatar
 
Join Date: Mar 2007
Location: Switzerland
Default

A nice to have would be if you could right-click for example on a NPC and you could chose there the option "report bug". A window would pop up and there are then different options for the kind of bug (for example: the active quests and if its a vendor or a NPC with some other function these actions can be selected) and the problem shortly described. At the same time the variables that define the relation and the status between the NPC and you (also with respect to the quests) are attached to the report.

Something similar can also be used for other kind of bugs like menus that do not work properly or mobs that behave strange.

Such a kind of system would reduce the complexity of the bug report tool and you would not have to look through a big amount of possible options in a drop down menu. On the other hand this reporting system can also have bugs itself.
__________________
Albert Einstein:
"Science without religion is lame, religion without science is blind."

Scientia est Licentia - Knowledge is Freedom - Join our Science and Archaeology focussed command.
Admin of the German Stargate Worlds Wiki
Amenopophis is offline   Reply With Quote
Old 12-11-2007, 04:59 AM   #14
Rryst
 
Rryst's Avatar
 
Join Date: Feb 2006
Location: Aus
Default

Bugs - Forms.

Game issues - Forums.
__________________
Rryst is offline   Reply With Quote
Old 12-11-2007, 06:24 AM   #15
Khabarach
Moderator
 
Khabarach's Avatar
 
Join Date: Nov 2006
Location: Dublin, Ireland Guild: Scientia Est Licentia
Default

Quote:
Originally Posted by Khestral View Post
Ok so here is something I can work with

We talk about this stuff alot at work and I want your thoughts on it now... What do YOU think would work for this type of thing?

Tell me what your thoughts are on this
It's not quite a tool, but one thing that is very important is having lots and lots of communication and interaction. There's nothing quite as frustrating as spending time filling out the appropriate forms and sending them off only to hear absolutely nothing about the issue for weeks, leaving you wondering whether the forms are even read. By far the most common complaint I've seen in beta's is not 'x is broken', it's 'they aren't listening'. A bug database as mentioned would definitely help.
__________________

Khabarach is offline   Reply With Quote
Old 12-11-2007, 06:32 AM   #16
DOAM
 
Join Date: Oct 2007
Default

If you have a character limit for the "description" field, then please have a secondary field with no character limit, that dev's and read/ignore at their leasure. That way when really bored people, such as myself, start testing the bug, to see what/where/if/how/why/whatever causes it, and what doesn't, we can list all the random info. Instead of running out of space to type, after two or three sentences.


Oh yeah, a nice part of the tool would be a "search" function, so that you could search through all the currently "queued" bugs, and click on box on that bug, which would "link" your bug report to another. Not sure how many people would use it, but even if a few did, it'd help immensly.
DOAM is offline   Reply With Quote
Old 12-11-2007, 06:46 AM   #17
Penguin
Moderator
 
Penguin's Avatar
 
Join Date: Oct 2006
Location: Australia
Default

From someone who has never really beta tested anything before I guess I can bring a different perspective.

Ease of use should be the number one priority, both for us and you QA folks. We have to want to report bugs, it has to be that easy.

It could be fun. We're all provided with an aerosol can item "Bug Spray" which we use to "report" the bug, for example. (except what happens when the bug is with the bug spray.......)

Perhaps competitions could help.

Separate and clear channels for reporting bugs, and more subjective issues.

A way for us to know our feedback is being valued and utilised(and not a generic "You bug report is important to us" message).
__________________

Vice Chancellor of Scientia est Licentia, a science and archeology focussed guild, check out our forum or website.
Penguin is offline   Reply With Quote
Old 12-11-2007, 07:40 AM   #18
Amenopophis
 
Amenopophis's Avatar
 
Join Date: Mar 2007
Location: Switzerland
Default

Quote:
Originally Posted by DOAM View Post
Oh yeah, a nice part of the tool would be a "search" function, so that you could search through all the currently "queued" bugs, and click on box on that bug, which would "link" your bug report to another. Not sure how many people would use it, but even if a few did, it'd help immensly.
That's also something I wanted to add. The good thing about my proposed system is that it could do the search directly and list the bugs that are reported regarding a certain NPC for example. This would then eliminate already quite some searching time and the testers would probably make use of it more frequently. And as you said DOAM, it would be extremely helpful for the devs since they would not have to sort so many reports.
__________________
Albert Einstein:
"Science without religion is lame, religion without science is blind."

Scientia est Licentia - Knowledge is Freedom - Join our Science and Archaeology focussed command.
Admin of the German Stargate Worlds Wiki
Amenopophis is offline   Reply With Quote
Old 12-11-2007, 07:54 AM   #19
Spaceman
 
Spaceman's Avatar
 
Join Date: Feb 2006
Location: I dont know. You Tell Me
Default

Forums for Beta players. So they all can chime in on what they think is wrong with the game.


And in game bug reporting tool. So Beta players can report on whats broke. I think that is all thats needed.
__________________
Spaceman is offline   Reply With Quote
Old 12-11-2007, 08:27 AM   #20
SnakeCharmer
Banned
 
Join Date: Feb 2006
Location: The Castle
Default

Quote:
Originally Posted by DOAM View Post
Oh yeah, a nice part of the tool would be a "search" function, so that you could search through all the currently "queued" bugs, and click on box on that bug, which would "link" your bug report to another. Not sure how many people would use it, but even if a few did, it'd help immensly.
SWG used to have something similar to that in the live game (insert joke here). You couldn't link to a previously filed report (which is a fantastic idea, btw), but you could search and find 'known issues' or similar occurances.
SnakeCharmer is offline   Reply With Quote
Old 12-11-2007, 09:29 AM   #21
FreejackeR
 
FreejackeR's Avatar
 
Join Date: Jun 2007
Location: Newcastle, England.
Default Beta toolio's

hmm interesting....

Well first of all thanks CME for asking us and allowing to trying a help out any which way we can, I know some of these might be implemented and some might not but at least the Developers are always looking which I find refreshing and also reassuring about the game we all hope to play when it is complete.

here are a few suggestions :

1.) 3 different types of beta ;

I) Technical - making sure things work as they say they do and making sure any hardware issues are addressed, also trying to do more of what happen if we change the resolutions from a to b,c,d,e,f,g,a,b,e,g,f,a?/ creating and deleting characters / trying to bypass the operational start up icons and starting from main file folder / trying to bypass the name filter, that kinda thing not game play more testing the actual systems side of it. (reporting would play a part in this beta so we can get the best things for the game we are going to play and pay for )

II) Stress test Beta - this is your bog standard let 5k people about the SGW and let them create havoc hehe, resetting characters, locking people into one zone and seeing what happens if 5k people turn up and can the servers coup with it, the general things we do in beta really play the game report the bugs.

III) Mission Beta - This would be select people from Technical & Stress test beta or even just select members from the forums or even people just signing up to that part of the beta. This beta would enable CME to ask set group of people to do set missions or tasks to see if the in game system works and how it maybe improved or doing bugged missions / testing fixed bugged missions or doing things that most people wont see until later in the game, also major reporting on this and this would mean people didn't report didn't continue in the beta.

I like the input from Blkjak and others who have input so far with the Reporting forms, and also the standard idea of Forums passworded or protected and of course the NDA(Non disclosure agreement)<-[encase anyone didn't know] people would have to sign up for, but I would like to see people removed from beta for breaking this agreement as too many times we see people breaking it and no action taken.

2) A private poll for beta users/testers in which people would vote and that would be there say and that it, no reply function just options to vote on good/bad, yes/no/maybe, Green/red/Blue/Yellow, things like that so the Developers get instant feedback with out all the hassle of reading through 400 posts just to see one part that hits the spot on target.

3) Developer TS2/Ventrillio Discussion with select members who have technical issue with the game aka hardware/software to resolve any problems, Or even just a open place where we can go and chat to Developer about issue we have, set times would be allowed for specific area's.

4) Developer TS2/Ventrillio Q&A on problematic area's / questions where people would respond threw IRC or selected forum area, so the CME team could keep control of the discussion and also have the desired effect of communicating with its beta testers.

5) Developer TS2/Ventrillio Feedback on how we are doing as testers and what we need to do to help the CME Developer teams and not just a case of us running through the game and then stop playing when the game comes out.

6) Changing the cover sheet on the TPS reports hehe Office space rocks! hehe if you ain't seen it WHERE HAVE YOU BEEN???? UNDER A ROCK? and you know the TPS report sheet is over due for a change. Q&A guys are going to love me

Just a few thoughts.

FreejackeR -out.
__________________
FreejackeR is offline   Reply With Quote
Old 12-11-2007, 10:05 AM   #22
Drake Brimstone
 
Join Date: Nov 2007
Location: Renton, Wa
Default

I would like to see in ingame bug reporting tool, as you may forget half the bugs you intend to report if you have to wait till you log out to report them on the forums. However, I believe it is important to also have a Web Based bug reporting tool in case the in game bug reporting tool is broken/bugged. I've beta'd a game where they had an in game bug reporting tool that crashed the client!

Definatly keep the tool's interface simple, checkboxes for yes/no, dropdown catagory listings, and ALWAYS include an "other" option!
Autofills for things like Race/Class/Level, Location, and the X number of lines from a system message log or something like it.


Also, I feel a quest rating system is a good idea. I don't, however want to see it every time I create a new character on every single newb or tutorial quest. Perhaps a flag that tells the system when you've filled out the survey once on the account making it less likely to get the survey on a different character on the same account. Certainly remove the survey from quests you've gotten enough feedback on. For a beginning tutorial wait till the tutorial is completely over before prompting for feedback, don't go asking us for feedback at each stage!

I agree that background gathering info for Chacter Race/Class/Level and Group composition could help deturmine if the quest was being completed by a group that was too high/low for the quest's intended difficulty.
__________________
Sam: "Didn't that tape say there were no fish in your pond?"
Jack: "Close enough."
Drake Brimstone is offline   Reply With Quote
Old 12-11-2007, 10:58 AM   #23
saltdav
 
saltdav's Avatar
 
Join Date: May 2006
Location: Westland, MI
Default

Quote:
Originally Posted by SnakeCharmer View Post
Heh.

Funny you should ask. I started typing something up not too long after I made the post. Be warned though, it's very, very rough, and not knowing details about the game and how it works, I could only make something that in of itself is very broad and general:

Bug reporting tool (rough draft)
1) Character name automatically put in
2) World waypoint: automatically put in, or easily obtainable by the player via UI
3) Is this quest related? Yes / No. (If yes, then drop down to name of quest from characters quest log NOT every quest in the game). If no, then skip to 4.

3a) Drop down listing name of quests in user journal
3b) Is the bug repeatable? Drop down Yes/No
3c) Description (with character limit)
3d) Submit, with autogenerated response / confirmation of submission email (ingame as well as out of game) to the player with a confirmation number for reference.
4a) Nature of bug/incident. Drop down menu: Character/avatar issues, World Geometry, Abilities/specials, World interaction (switch not working, npc unresponsive, etc), Exploit (dupes, etc), GUI issues, etc. Each drop down could/should have subtopics for further refinement.

The goal here is to narrow down the issue to determine which team it should go to for resolve. Ideally, all teams should be divided up (art, world builders, etc even if only on a broadcast email group) and bug reports parsed to the relevant dev teams if at all possible, or alternatively (and probably the best avenue), all bug reports should go first to QA for immediate follow up and replication, then sent to dev teams responsible for that area. QA should flag each report with a score indicating severity and urgency. The two are not directly related; a bug may be severe, but not urgent, and vice versa (example: a dupe exploit may be simple to replicate/fix, but its urgency is paramount). If QA is unable to replicate, then someone from QA should contact the person who submitted the bug for further follow up.

4b) Description (with character limit) - You don't want too much to where the info becomes convoluted and useless, but you don't want too little so that you can't get an accurate description.
5) Submit, with autogenerated response / confirmation of submission email (ingame as well as out of game) to the player with a confirmation number for reference.

I would argue that the drop down menus need to be as thorough as possible to help YOU guys isolate the issue as quick as possible. The problem for you guys is that players WANT to submit a bug, but at times it can be overwhelming trying to accurately describe what is happening. In beta invites, I'd stress the importance of a pen/paper to keep notes, or alternatively have a /notepad command in game that players can use in leiu of (it'd be nice if this could somehow be attached to the bug report...).



Quest Surveys:
Pretty easily done. POTBS had a pretty nice one although at times it became a nuissence. The biggest problem with surveys is you depend TOTALLY on your players to be honest and unbiased, both to get an accurate answer as well as us not just answering <whatever> to get it out of the way. Too many surveys or too many questions and we tend to have our eyes gloss over and click random numbers and get it over with.

The magic number of questions for <me> is three...
1) Did you find the quest entertaining? On a scale of 1 to 5 (with 5 being very entertaining), how would rate this quest?
2) Did you find the quest challenging? On a scale of 1 to 5 (with 5 being very challenging), how would rate this quest?
3) Did you find the reward for the quest be worth the risk and/or time involved? On a scale of 1 to 5 (with 5 being 'worth the risk/time'), how would rate this quest?


It would do wonders for you guys to have a background process that snapshotted how many people were in the group, what levels they were, and what classes they were, as well as the level/class the person filling out the survey was. It would give you a reason as to why someone thought a quest was too easy (too high of level versus the intended level of the quest, was it an intentional solo quest that the player did with a full group? Did the player try a quest that was intentionally set for a group/higher level? This is, of course, dependent on whether or not NPCs will give out quests to players outside the level range of the quest line the NPC is giving...)

You could possibly have an optional 'Comments' section where players can freely put thoughts about the quest down, with character limit. They don't have to fill out the comments to submit your survey answer, to avoid getting stuff like adkjfhakjhsdfkhaskdf and 'submit'.



Probably more than what you wanted, but there you go...
What he said!
__________________
"Hail to the King, Baby!" Ash - Army of Darkness
saltdav is offline   Reply With Quote
Old 12-11-2007, 11:56 AM   #24
MinganBlack
 
MinganBlack's Avatar
 
Join Date: Feb 2006
Location: Somewhere in the USA
Default

All I can say is make the reporting easy, for both you to read and sort (database for you). Also simple to fill out, complicated systems don’t work out to well. Check boxes work well this way you don’t have 5 million forms

1) Character name automatically & stats are automatic

1. Chose all that apply with issue:
a. Quest ()
b. Interface ()
c. geometry ()
d. GUI issues ()
e. Exploit ()
f. World interaction ()
g. Other ()

2. Description of Issue: In Detail (box)
3. Repeatable; Yes () No ()
4. Location of issue: In Detail( box)
5. Journal of what happened: What were you doing in the game and out of the game?(computer stuff did it )

6. Computer status
a. Froze up
b. Crashed
c. No issue
d. Slow
e. Other In Detail( box)



Oh also a Escape command!!! we can use if we get stuck in a wall or tree. I have gotten stuck before and had to wait hours till it was corrected. I would rather start over at a home site than get stuck in a wall for hours.
__________________
Minganblack

Maybe next time, you'll just wait and see."
"And blow my last chance of ever being right?" -- Jack and Jacob, Enemies

Last edited by MinganBlack : 12-11-2007 at 12:05 PM. Reason: Addtion
MinganBlack is offline   Reply With Quote
Old 12-11-2007, 03:02 PM   #25
blkjag
 
blkjag's Avatar
 
Join Date: May 2007
Location: San Francisco Bay Area, CA
Default

Adding more to this.. put a limit or threshold on the number of times a certain bug is reported by beta testers for the same problem along with dev flag and alert system that tell the beta testers that the dev is aware of the problem and the status of it inside the game and in the beta forum, The dev flags can be color coded, like blue for awareness, green for completed, yellow testing, red - being work on -

Example a certain NPC does not give you a quest - after a certain number of beta testers reported it of let say 200 out of 500 beta testers, there is no need for the reminding beta testers to keep reporting the same bug until after a fix been done and you want to test it out. You had enough of the population input on it where you can close it, and evaluate it accordingly.

This way it will save both parties time, also, it will help you become aware of the more critical bugs faster, if you set critical threshold numbers based on number of times the bugs is being reported by the beta testers population, like if more than 35% of beta testers is reporting the same problem then the dev alert will let you know the frequency and allow you to prioritizes the critical bugs to work on.

Now how durable to do a system or provide something like this I do not know.
__________________
Encourage Positive Behavior and GamePlay
-
Dial the Gate, Let the Adventure Begin !!!

Last edited by blkjag : 02-28-2008 at 02:41 PM.
blkjag is offline   Reply With Quote
Post New Thread Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -7. The time now is 06:55 AM.


Stargate WorldsCheyenne Mountain Entertainment FireSkyStargate SG-1Stargate AtlantisMetro-Goldwyn-Mayer Studios Inc.

Cheyenne Mountain EntertainmentTM is a trademark or registered trademark of Cheyenne Mountain Entertainment, Inc. in the U.S. and/or other countries. Stargate SG-1, STARGATE ATLANTIS, and STARGATE WORLDS are trademarks of Metro-Goldwyn-Mayer Studios, Inc. All rights reserved. All other trademarks or tradenames are the property of their respective owners. Statements herein concerning future events and developments and the Company's expectations, beliefs, plans and estimates constitute forward-looking information that involves risks and uncertainties. Cheyenne's actual results could differ materially from those indicated by the forward-looking information.

Stargate SG-1 TM and © 1997 — 2006 Metro-Goldwyn-Mayer Studios, Inc. All rights reserved.