More Consumer Beta Testing

This is a post from the flight-sim group. Change a few names and it sounds like a perfect description of the MS:CPR situation. ---------- Newsgroups: comp.sys.ibm.pc.games.flight-sim Subject: Paid to Beta test! Is this the future of *** companies? Date: 14 Dec 1997 06:06:05 GMT For those looking for a financial opportunity this is not the post you are looking for! The title of this post is refering to the recent practice of just selling the product and then getting the customers to help test and fix it! Case in point, Red Baron2. I have looked forward to this title for nearly 2 years now. Since that era of aviation has been my hobby for over 35 years and seeing as RB1 was SOOOO enjoyable the anticipation for this game was positively excruiating as it was delayed time and again. But now we have finally gotten the game. Paid a pretty penny for it too! I've been a beta tester on three other flight sims and have seen how quickly a large tester base can find even the most innocuous errors. Simply because a large number of differant people are doing a variety of differant things on widely diverse hardware. Most companies get thier testers together, give them the software and then let these people tell them what doesn't work. Then they fix it all and box the finished product and sell it. ( the ususal compensation to the testers is a free copy of the finished product). However Sierra seems to be doing it with a slight twist. They boxed the product( Just in time for Christmas!) and sold it. Now we are testing it and telling them all of the things that don't work so they can make a patch ( which they will be nice enough to D/L to us for free) . From what I have seen on their boards and from my own experience, at least 95% of the problems would have been noted and reported in the first 3-4 days of a properly done OUTSIDE beta test! From some of what I am seeing here it looks as though Sierra isn't the only taking this approach. Is it time for some class action law suits to bring the companies back to the concept of fixing it BEFORE they ship it? ---------- BTW, I haven't tried RB2 myself yet, but it looks like it is surprisingly good (decent frame rate w/o 3D, minor bugs, mixed decision on the flying model) in spite of bad pre-release press, opposed to the currently disappointing CPR 1.0 (with its glaring flaws [AI, frame rate]) and all of the hype that surrounded its release. Before you send me UCE, I know what you're thinking... Did he complain to five or six postmasters last month? Now, you must ask yourself one question: "Do I feel lucky?" Well, do you, punk?

This is nothing new in the software industry. Look at Quake II for another example. It is truly hard to tell if the people involved in these cases knew what was wrong or it was an honest mistake. We can thorize as much as possible but the bottom line is it happens a lot in the software industry. I don't like it as much as others don't But as loing as software sales are not significantly hurt it will continue to happen. Go read the interviews on www.pcgames.com of the "So you want to be a game developer". A couple interviews the devlopers talked about releasing a game and then having patches. Thanks John

Quoted text (63 lines)

> This is a post from the flight-sim group. Change a few names and >it sounds like a perfect description of the MS:CPR situation. > >---------- > >From: [email hidden] (PSPiranha) >Newsgroups: comp.sys.ibm.pc.games.flight-sim >Subject: Paid to Beta test! Is this the future of *** companies? >Date: 14 Dec 1997 06:06:05 GMT > >For those looking for a financial opportunity this is not the post you >are looking for! > >The title of this post is refering to the recent practice of just >selling the product and then getting the customers to help test and >fix it! > >Case in point, Red Baron2. > >I have looked forward to this title for nearly 2 years now. Since >that era of aviation has been my hobby for over 35 years and seeing as >RB1 was SOOOO enjoyable the anticipation for this game was positively >excruiating as it was delayed time and again. > >But now we have finally gotten the game. Paid a pretty penny for it >too! > >I've been a beta tester on three other flight sims and have seen how >quickly a large tester base can find even the most innocuous errors. >Simply because a large number of differant people are doing a variety >of differant things on widely diverse hardware. > >Most companies get thier testers together, give them the software and >then let these people tell them what doesn't work. Then they fix it >all and box the finished product and sell it. ( the ususal >compensation to the testers is a free copy of the finished product). > >However Sierra seems to be doing it with a slight twist. They boxed >the product( Just in time for Christmas!) and sold it. Now we are >testing it and telling them all of the things that don't work so they >can make a patch ( which they will be nice enough to D/L to us for >free) . From what I have seen on their boards and from my own >experience, at least 95% of the problems would have been noted and >reported in the first 3-4 days of a properly done OUTSIDE beta test! > >From some of what I am seeing here it looks as though Sierra isn't the >only taking this approach. > >Is it time for some class action law suits to bring the companies back >to the concept of fixing it BEFORE they ship it? > >---------- > > BTW, I haven't tried RB2 myself yet, but it looks like it is >surprisingly good (decent frame rate w/o 3D, minor bugs, mixed >decision on the flying model) in spite of bad pre-release press, >opposed to the currently disappointing CPR 1.0 (with its glaring flaws >[AI, frame rate]) and all of the hype that surrounded its release. > >Jeff Vincent mailto:[email hidden] [link lost in import] >Before you send me UCE, I know what you're thinking... Did he complain >to five or six postmasters last month? Now, you must ask yourself one >question: "Do I feel lucky?" Well, do you, punk?

Quoted text (9 lines)

> This is nothing new in the software industry. Look at Quake II for another > example. It is truly hard to tell if the people involved in these cases knew > what was wrong or it was an honest mistake. We can thorize as much as > possible but the bottom line is it happens a lot in the software industry. I > don't like it as much as others don't But as loing as software sales are not > significantly hurt it will continue to happen. Go read the interviews on > www.pcgames.com of the "So you want to be a game developer". A couple > interviews the devlopers talked about releasing a game and then having > patches.

Well I have to reply regarding Quake2 ... I'm not the most demanding Quake player, but Quake2 worked for me out of the box, first time. The frame rate is good. The graphics are beautiful. I'm ***ed. And I have yet to find any serious bugs that eliminate game play. Let me compare that with MS:CPR ... I already returned CPR ... it was not playable, at least on my system. Doug p.s. I am NOT denying that Q2 has bugs ... but unless you have it, you won't appreciate how much more "done" it is than CPR ...

Thanks for the excellent post (and re-post), Jeff. Although this concept of shoving a product out the door *well* before its time may be nothing new in the software industry, it stank, it has always stunk, it will always stink, and it stinks now! Thanks again for the perspective... Bart

I agree entirely. I didn't see the original posting so I don't know what product you were refering to. However, this is exactly the issue I mailed back to Europress about their recent "International Rally Championship", which I believe they rushed out the door in order to make the Christmas sales period.