Separate Quality Testing from Availability

January 11, 2018 / Steve Johnson

Quality testing identifies anomalies or defects that are not working as expected. Typically, quality testing—both alpha and beta phases—are the responsibility of the development team and/or quality assurance team.

“Beta” is one of those amorphous terms that we use to mean many different things. Here are some crisp definitions.

“Alpha” is a testing phase where a release candidate (or potentially shippable increment) is tested on client systems (such as devices and browsers) and data by VENDOR employees. “Beta” is a testing phase where a release candidate (or potentially shippable increment) is tested on client systems and data by CLIENT employees. By this definition, more companies should be doing ALPHA than BETA as most customers are simply terrible at running comprehensive tests.

In general, these test phases should be administered by QA or development. A product manager declares the end of beta by determining there are no critical issues with the software that will impact customers.

However, “end of beta” isn’t necessarily “General Availability.”

There are three phases of availability: 1) Invitation, 2) Limited, and 3) General. Invitation availability is determined by a product manager for a very small number of customers (5 to 10) to implement the product. In this phase, the product manager provides support and no sales people are involved. Limited Availability is a broader set of clients and prospects who fit a desired profile; the product manager provides support and serves as a sales engineer to a small number of sales people. General Availability is when the product is available to all prospects and customers.

An example of the three phases is how Amazon introduced Echo. First it was available only by invitation, then only to Amazon Prime members. Now of course it’s available to anyone.

These phases of availability ensure that both the product capabilities and its promotion are “ready.” We know the top features that excite clients; we know they work in production; we have nailed the price, we can take an order.

To quote my friend Saeed Khan, you need to “Nail it before you Scale it.” Invitation and Limited Availability help you Nail it; General Availability is for when you’re ready to Scale it.