I’ve been meaning to start writing more about startup-related stuff – how we get product out, sales and marketing idea/issues/challenges, funding stuff, etc. – in addition to the email-related articles. Oh, and also toss in some fun stuff about living in the Bay Area like how to eat at Gary Danko last minute or where to get an excellent lunch for under $3. I’ll get back to the food stuff soon, but right now I’d like to kick off the startup posting with some thoughts on "Beta" as we are beginning our ClearContext IMS 3.0 Beta program.
Back in my enterprise software days, beta was pretty clear. We’d develop stuff internally, test it, then get everything ready to release and before officially announcing the release, we’d have a small group of customers deploy the software. In this process, we’d generally find a few bugs and maybe clean up some APIs, expose a couple more things, etc.
At ClearContext we follow a pretty similar path. Once a new version is more-or-less working, we start using it internally. At a certain point, we lock down the feature set, stop doing development on new features and start doing formalized testing and bug-fixing on the feature-freeze version of the software. Once we’ve tested on a number of platforms and fixed all the major bugs we can find,
we move to an "alpha" which we give to a small group of people. At this point, there are a number of minor bugs left in the software, the occassional major bug, and a lot of usability and fit/finish things to tune. We use this period to figure out what fine tuning needs to be done to the product to move to a released "GA" (general availability) production release. After making these changes, we release a "beta" version of the software. This is more or less the final release, but by putting it in the hands of a wide range of users with all sorts of different environments who use the product in all sorts of different ways, we generally find a few more bugs here as well as some UI/usability suggestions. We address those issues, then put out a final "release candidate" as a final check prior to marking it our "production release."
This type of progression was for many years more or less standard in the software industry. People generally had a good idea of what level of stability/polish/etc. to expect from "alpha" "beta" and other such pre-release products. However, with the advent of web-based software and services, everything has changed. Beta now can mean anything from "we’re tossing out some crap we threw together yesterday, not sure if it works" to "this service has been running for a very large user base and is really production software, but we haven’t decided on our pricing/revenue model yet or our final feature set, so we’re just calling it beta until we figure all that stuff out."
And this is where things start to become a potential problem as I see a lot of this trend spilling over into non-web based software products as well. Because now all of a sudden, "beta" can become an excuse to put out shoddy, bug-ridden product that is really more an ad-hoc market test than anything else. For companies like ours that still deal in real product that we sell to people on a traditional software basis (as opposed to an ad click revenue model or something), this is an important step in the product development process, and the blurring of "beta" lines definitely makes things more confusing to a lot of users out there.
In my next entry, I’ll talk in more detail about how we run our beta process at ClearContext and some of the things I see in "beta excuse" programs out there that I think do nothing but confuse and irritate users.
In the meantime, here are a couple of interesting reads on the topic: