Longtime readers of the blog have been eagerly waiting on this post for quite some time, as it has been almost two years since I wrote part 1. Sorry to keep you waiting so long! In that post I wrote about some of the frustration and confusion around the use of the term "beta" in the software release process, largely due to the way it has been used in the web 2.0/SaaS world. Brad Feld just wrote a post bringing up the subject that largely echoes this key point from my previous post (and questions whether "Private Beta is Bullshit"):
And this is where things start to become a potential problem as I see a
lot of this trend spilling over into non-web based software products as
well. Because now all of a sudden, "beta" can become an excuse to put
out shoddy, bug-ridden product that is really more an ad-hoc market
test than anything else. For companies like ours that still deal in
real product that we sell to people on a traditional software basis (as
opposed to an ad click revenue model or something), this is an
important step in the product development process, and the blurring of
"beta" lines definitely makes things more confusing to a lot of users
out there.
"So – why not get rid of the "private beta" label and call all of these things alphas." he asks. Well, because they often aren’t alphas!
I think this issue is actually about two very distinct points.
One point is Brad’s key point (and I made in 2006, the WSJ made in 2005, and zephoria made in 2004) – the fact that the way this term has been used has largely made the term meaningless, and "betas" now range from bug-ridden early alphas to true beta quality almost ready for GA software. Brad’s post has a lot of great comments, including a couple that point out "beta" is also sometimes used as a marketing tag to denote "new" features – even after they are in widespread general release.
There’s a second point, though, and a valid reason for private betas to exist (funny timing on Brad’s post, as our ClearContext Personal product is currently in, you guessed it, private beta). In the old world of disks and distribution through traditional channels, you as the developer were able to control who you gave access to the beta and push out in waves you selected. With both web-based apps and software that uses the web as its marketing and distribution channel, the way a lot of companies reach their users is by putting something out on the web. A number of Brad’s comments describe scenarios where a limited beta is a very useful and valid part of the process. And logistically, what’s now commonly referred to as a "private beta" is often the best way for companies to accomplish that. With our latest product, we did an alpha with a small existing group of customers. Then we did an initial beta, again with existing customers only. Next we opened our private beta to a few thousand users. I posted about the different types of beta testers and the part they play in our release process. On our corporate blog we posted specifics about what we gained from the private beta – the biggest thing was getting usability feedback from brand new users who had never seen anything like this product before.
A lot of companies still use beta as an excuse for taking shortcuts and not having a really rigorous development and release process, but the advent of web-based deployment and/or distribution of software has also somewhat changed the nature of the beta process. Perhaps companies will standardize on usage so people can have some understanding and expectation about what it means for something to be called a "beta" release, but I doubt it will happen without any external pressure. It would be nice to see some of the leading tech blogs call companies on this and evaluate whether their "beta" releases are really deserving of that tag. A "how close to ready for primetime is this app" rating would be a very useful service to their readers.