Agile Zone is brought to you in partnership with:

Frank was born in Ireland and now lives in the USA. His initial training was in Computer Science from Trinity College, Dublin (so they're really to blame for this). He also has a Ph.D. in Computational Neuroscience and after realizing that computers were still as dumb as a bag of hammers realized that he had better go earn some money and stop fooling around. Right now he likes to think he is a part-manager, part-developer, part-architect developing applications for a major telecommunications firm because "code monkey" just doesn't sound as good. Frank is a DZone MVB and is not an employee of DZone and has posted 16 posts at DZone. You can read more from them at their website. View Full User Profile

To Beta, or not to Beta - That is the question

12.31.2009
| 1542 views |
  • submit to reddit
"Whether 'tis nobler in the mind to suffer the slings and arrows of outrageous bugs, Or to take arms against a sea of defects, And by opposing end them." (with props to Bill S.)

In the past few years it's become more and more acceptable to release "Beta" software to the public - almost as if it was a production release.

The reasons for that I believe are manifold but boil down to
1) Gain user feedback
2) Release early to gain "mindshare" and/or get first to market.
3) Your QA process or team isn't that great (or your unwilling to spend money on them) and you'd rather have users do your testing for you.
4) You need to gain the confidence of your customers / investors in what you're building.

The number one proponent of the Beta, I must say is Google. I've been a Google Docs user for well over a year now and I love it - but it's still in Beta. The one thing I don't love is the FREQUENT UI changes in direction though - but hey it's a Beta and it hasn't lost a Document or Spreadsheet of mine yet.

But it's clear from using what Google call a "Beta" that their goals are really predominantly #1 (user feedback) and probably to a lesser extent #2 (mindshare). It's been very well tested (internally) before it hits the public. You can read more of what Google think about Betas here.

So the quality of their Betas is quite high (see also gmail) and a Beta for them is probably a production release for 90% of everyone else.

Here's the problem though - in making Beta's "popular" I think they (and others) have lowered the bar at least insofar as management can now look at it and say "Well if Google did it - why can't we?". So there's a tendency now to ship lower quality software and at the last minute people decide to slap a "Beta" label on it if it's not up to snuff so that they can declare victory. Then they hope people will use it as if it was production quality.

Personally I think software professionals should be using a public "Beta" ONLY as a last resort. If you need to gain user feedback - try not to be cheap about it. Get a bunch of potential users in a room and have them use your product - try not to prime them - just watch them. Video them. Understand what are they trying to do - how does that align with your product? Can you make your application so easy to use it doesn't need a manual? That should be the goal (although that might be ultimately unattainable any progress in making your application easy to use is good).

If the reason is #2 (to be first to market) then I think you don't understand technology trends. Neither Google or Internet Explorer (or Firefox) were first to market - look where Yahoo! and Netscape are now.

I guess I never really "got" the whole first-mover advantage thing. Think about it:
  • Apple vs. Windows (and back again)
  • Ford vs. Toyota
  • Betamax vs. VHS
  • Atari vs. Nintendo
  • WordPerfect / WordStar vs. Office
  • Friendster vs. Facebook vs. MySpace
The anecdotal lesson I got from these cases was to NOT be first mover and learn from the other guy and where he fails.

Anyway if you're in the position of #3 (you're not investing in your QA team/process) then that's a sorry state of affairs. Although we live in a world of finite resources - finite dollars and hard-to-find tech experts so I guess it can be excusable sometimes. But if you just punted to force your users to do the bulk of your testing work then shame on you.

If you're in position #4 - releasing a Beta to the public to gain confidence well it's hard to argue too much with that given the sorry state of "production" software these days (*cough* Vista *cough*) people's expectations are sadly low so its understandable people (especially investors, customers) want some reassurance.

Don't get me wrong on this though - I'm not the type of guy to wait until the product is perfect. It's more a question of what are your motivations and are you being lazy in releasing a Beta? If you want user feedback - GET USERS and GET THEIR FEEDBACK! Don't ship junk!

After doing some Googling I came across a blog article with similar sentiments
A call for revolution against Beta Culture

"I'm mostly tired about the fact that it seems that we all have given up. Tired because . . . . in reality, it's laziness and a poor job on the manufacturer part that we have accepted without questioning. Instead of calling foul play and refusing to participate, we keep buying."

"That's the key: We have surrendered in the name of progress and marketing and product cycles and consumerism."

Right on man - it just doesn't feel right. It feels lazy and short-sighted.

Here are some other thoughts by other folks

So here's my question to you, dear reader, what is your opinion on Betas? Useful software engineering technique or cop-out?
Published at DZone with permission of Frank Kelly, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)