Topic: What beta testers do!

[Pharacon]

Ok before i get flamed sum credentials for claiming to know what i know:
1) I try to sign up and test any game that has a public beta/alpha test, a notable one, sfc2 i help in (tho i never did make it past the dyna login screen =\
2)I actually got payed to QA with Sony Interactive in PSX games and PC ported games.

Ok now that that is there he is basically how Sony did the QA, i believe it's pretty much standard crap in the industry.

Step one) The game (or patch) is received, a lead tester (interface between the QA testers and programmers that make fixes) is given the game, he/she goes over it for a day or two getting all the obvious bugs written down and catalogued.

Step Two) A QA team is formed to test the game, in sony we got to play ones we wanted, or in the case of big blockbuster surefire games people were assigned.

Step Three) tester start to bang away at it, the go over in phases different parts of the game, like one period they are all testing single player, another multiplayer etc etc When they run across a bug normally they try to get it to duplicate easily and then write a step by step report on how to do it. This is very difficult sometimes because you might be sitting there and it just happens, and if its a major bug (crasher, or corrupter) you have to figure out how you did it. Some times this takes 4ever!

Step four) At sony we did it in weekly chunks or after a certain amount of bugs were reached (based on the bugs severity). Like they r not going to reprogram a game if they have two bugs that can only be reached if you turn off the sound start a single player game all the while punching a certain button, in a certain order. ( you may think i'm lying but sometimes that's what you test, no sound holding down y button all sorts of crazy things) After all that the lead tester sends the bug database over to programmers so they can come up with another REV (or revision) to start the program over.

All of this is done over and over till they reach a point where is has an acceptable (many games this is extremely flexible term *cough* gameday '98 *cough*) level, and has done a certain amount of "tester hours" now if your talking like 100 tester hours, this is the amount of time it has been tester so if tester A tests the game 8 hours and tester B does it for another 8 the game has completed 16 hours, this is reset every rev that is received.

OK i hope this helps out a little bit for everyone here that doesn't understand how the QA process works, and believe me with a thing like this the longer it stays in the QA department the better it is! Just to let you know some things i tested stayed in the department less then 12 hours! (the whole department was pounding it) the longer you take the more likely you are to find problems.

-Pharacon

PS: See all you guys on the triangle ROMULANS KICK BUTT NOW WAIT TILL I GET SUM WORKING WEAPONS I'll be unstoppable!  

[Credo Narth]

Yeah... but at what point did the SFC3 beta testers say to each other, You know what, this game really sucks the big one. Let's get some donuts and beer and load up EAW/ OP/ Deanna Troi Does Dallas?

As this game stands, I couldn't recommend it. I do, however, strongly recommend EAW. Curiously, in the UK, you can't buy EAW from ANYWHERE. Nor OP... Not in shops, not online, not even in garage sales. But already you can buy SFC3, second-hand, for a tenner (that's 15 of your US bucks, sorry, 16 now that Bush is driving the economy to ruin). Says it all, really.

[cherokee158]

Thankyou. I was interested to learn what QA is like. It's amazing how a good manager can make even something as fun as a good game an exercise in tedium :-)

Still, it beats hod carrying...

I disagree with the last post, though, to a degree. I think SFCIII isa good GAME ENGINE in search of a game. I infinitely prefer this game engine over the previous efforts, which were often flaky in multiplay and had poor (although much prettier) interfaces. The new cloak,  warp and ship building features are great. I think if this engine is fleshed out later with more ships and races, and maybe a little more ambience (how can you do music and UI eye candy so right in SFC II and then just toss it out?)  it will be a must have.

Looking forward to SFC4. Until then SFC3 will do.

[FEDX-Master Zen]

well i hate to say it, but if you say look foward to sfc4 and sfc3 has only been out 4 months then you know it SUCKS!  heck the first day sfc3 came out people were talking about sfc4 on the forums.  

[cherokee158]

I wouldn't be that harsh. I dont think SFCIII sucks. I have had a lot of fun playing it. At least as much fun as I did the prior games. I just think it has room to grow. It's a game that could've easily have spent another year in development, adding the chrome. But I'd just as soon have it now and an expansion later as wait, I guess.

After all, I only had SFCII a few months before I decided that playing patch roulette wasn't worth the headaches. I have yet to find a computer game good for more than a year. Even if the game play is marvelous, rest assured some inevitable upgrade will break it.

To be honest, I sort've miss board games these days. I have yet to toss one out because it was incompatable with my new table...

(although they are not always compatable with my kids...talk about missing data and crashes!)



 

[Pharacon]

Ok before i get flamed sum credentials for claiming to know what i know:
1) I try to sign up and test any game that has a public beta/alpha test, a notable one, sfc2 i help in (tho i never did make it past the dyna login screen =\
2)I actually got payed to QA with Sony Interactive in PSX games and PC ported games.

Ok now that that is there he is basically how Sony did the QA, i believe it's pretty much standard crap in the industry.

Step one) The game (or patch) is received, a lead tester (interface between the QA testers and programmers that make fixes) is given the game, he/she goes over it for a day or two getting all the obvious bugs written down and catalogued.

Step Two) A QA team is formed to test the game, in sony we got to play ones we wanted, or in the case of big blockbuster surefire games people were assigned.

Step Three) tester start to bang away at it, the go over in phases different parts of the game, like one period they are all testing single player, another multiplayer etc etc When they run across a bug normally they try to get it to duplicate easily and then write a step by step report on how to do it. This is very difficult sometimes because you might be sitting there and it just happens, and if its a major bug (crasher, or corrupter) you have to figure out how you did it. Some times this takes 4ever!

Step four) At sony we did it in weekly chunks or after a certain amount of bugs were reached (based on the bugs severity). Like they r not going to reprogram a game if they have two bugs that can only be reached if you turn off the sound start a single player game all the while punching a certain button, in a certain order. ( you may think i'm lying but sometimes that's what you test, no sound holding down y button all sorts of crazy things) After all that the lead tester sends the bug database over to programmers so they can come up with another REV (or revision) to start the program over.

All of this is done over and over till they reach a point where is has an acceptable (many games this is extremely flexible term *cough* gameday '98 *cough*) level, and has done a certain amount of "tester hours" now if your talking like 100 tester hours, this is the amount of time it has been tester so if tester A tests the game 8 hours and tester B does it for another 8 the game has completed 16 hours, this is reset every rev that is received.

OK i hope this helps out a little bit for everyone here that doesn't understand how the QA process works, and believe me with a thing like this the longer it stays in the QA department the better it is! Just to let you know some things i tested stayed in the department less then 12 hours! (the whole department was pounding it) the longer you take the more likely you are to find problems.

-Pharacon

PS: See all you guys on the triangle ROMULANS KICK BUTT NOW WAIT TILL I GET SUM WORKING WEAPONS I'll be unstoppable!  

[Credo Narth]

Yeah... but at what point did the SFC3 beta testers say to each other, You know what, this game really sucks the big one. Let's get some donuts and beer and load up EAW/ OP/ Deanna Troi Does Dallas?

As this game stands, I couldn't recommend it. I do, however, strongly recommend EAW. Curiously, in the UK, you can't buy EAW from ANYWHERE. Nor OP... Not in shops, not online, not even in garage sales. But already you can buy SFC3, second-hand, for a tenner (that's 15 of your US bucks, sorry, 16 now that Bush is driving the economy to ruin). Says it all, really.

[cherokee158]

Thankyou. I was interested to learn what QA is like. It's amazing how a good manager can make even something as fun as a good game an exercise in tedium :-)

Still, it beats hod carrying...

I disagree with the last post, though, to a degree. I think SFCIII isa good GAME ENGINE in search of a game. I infinitely prefer this game engine over the previous efforts, which were often flaky in multiplay and had poor (although much prettier) interfaces. The new cloak,  warp and ship building features are great. I think if this engine is fleshed out later with more ships and races, and maybe a little more ambience (how can you do music and UI eye candy so right in SFC II and then just toss it out?)  it will be a must have.

Looking forward to SFC4. Until then SFC3 will do.

[FEDX-Master Zen]

well i hate to say it, but if you say look foward to sfc4 and sfc3 has only been out 4 months then you know it SUCKS!  heck the first day sfc3 came out people were talking about sfc4 on the forums.  

[cherokee158]

I wouldn't be that harsh. I dont think SFCIII sucks. I have had a lot of fun playing it. At least as much fun as I did the prior games. I just think it has room to grow. It's a game that could've easily have spent another year in development, adding the chrome. But I'd just as soon have it now and an expansion later as wait, I guess.

After all, I only had SFCII a few months before I decided that playing patch roulette wasn't worth the headaches. I have yet to find a computer game good for more than a year. Even if the game play is marvelous, rest assured some inevitable upgrade will break it.

To be honest, I sort've miss board games these days. I have yet to toss one out because it was incompatable with my new table...

(although they are not always compatable with my kids...talk about missing data and crashes!)