The Art of Beta Testing

What is a Beta and Why is it Used?

A Beta version is originally used to refer to software applications that are close to completion. Beta software is still under active development, debugging, and revisions. Beta software typically doesn’t have all features (even sometimes critical ones) implemented fully or at all.

In the modern terminology, one can have a beta. In this usage, the beta represents a time span during which public users are selectively invited to try out unfinished or untested software as it under goes its final development stages.

Betas give companies a chance to do a controlled semi-public test of their software and to build buzz/hype prior to launch. 100+ users can find far more bugs and problems, faster, than any development team can ever hope to. They also serve to stress test unproven hardware and software systems; safe-guarding against potentially disastrous launch day failures.

Betas are nice and safe testing grounds because users are infinitely more forgiving and helpful about flaws and bugs when participating in a beta than with a full/live service. Problems discovered in a beta typically won’t haunt a service’s reputation, unlike problems discovered during full release.

Types of Betas, Closed and Open

There are two types of Betas. Closed Betas are where public users can only access the service or application if they have been invited. Open betas have no such access restriction. Regardless of the service offered, betas are ALWAYS free. No one useful or moral will pay to be in a beta. Closed betas almost always have an NDA, open betas never do.

Timing

An average closed beta for software that has 4 years of development behind it is 4-6 months long. Realistically, anything less than 1 month is unlikely to be very productive.

Betas typically end with the software or service launching. This is the way it’ll be assumed by users to be; sometimes deviating from this part of the formula expectation can cause problems. Example: “Man that stuff was so busted when the beta ended, I’m not using it and you shouldn’t either.”

Some Unwritten Rules

The service being tested will not always work, will have bugs, and will not always be available to the beta testers. If usage windows are being used, users expect to be able to find the times when they can use it to be easily accessible. This is all normal and users generally don’t hold it against the company.

Data, profiles, and anything user related is subject to total destruction at the whim of the company. Users don’t expect anything of theirs to survive the beta. However, if the community has really poured its heart out and generated a lot of content, then it would be a big no-no to delete their hard work.

Users who were able to use the beta for free will have to pay to keep their accounts and services when the service goes live. This is fine and expected. They need to be able to keep their accounts however as account = identity.

How to Host a Beta and How to Use it to Your Advantage

You have enough once the service or application is nearing completion, and its mission critical systems can be used. So long as users can get into the service and get basic value from it, that’s enough. Ideally, one can start the beta when one starts the real pre-launch marketing.

Start With a Closed Beta, Always

Send out invites, talk to the community, promo on blogs. Get people interested and give the cream of the crop special access to start using the service for free. After the closed beta is working mostly well, then an open beta can be considered. Depending on what kind of site or application you are testing, you could just end up with a bunch of worthless 12-18 year-olds who do nothing but complain, slander, drive away the really cool early adopters, and then jump ship when your service goes to pay.

Talk to Your Beta Testers Constantly

You will get a flood of advice, much bad, some good, some golden. Identify who really knows their stuff, and who others respect. These are the initial community pillars and it is a priority to keep them happy and on-board. If you must, you can pay someone and essentially ‘seed’ the starting community with someone who toes the company line. Just be sure to make sure they don’t get found out as a fake.

Be Responsive and Open

When dealing with the beta community, addressing their issues is important. If your company comes across as a cool-headed, responsive, responsible, empathetic, and fair, then the good faith built during the beta will be a foundation from which the community grows on. Remember there’s an NDA and hide nothing from the closed beta people. Honesty, accountability, and transparency count for tons in the company-community relationship. When the NDA expires and the beta goes live, you want these people to sing glowing praises behind your back.

Buzz Within Buzz

Talk constantly to the beta testers about what you are doing with the service, where it is going, what issues are being addressed, what its time-line is, what its future is. Don’t stop doing this when the beta ends. In fact, don’t stop doing this EVER. Update daily if you can, weekly at the worst. Keep their eyes and ears looking forward, always.

Do something Special for the Closed Testers.

Run events, contests, give-aways. Give-aways are excellent chances to target a few free months of use to those important community leader types that have appeared during the beta. Help make sure they stay on board for that delicate post-launch span of time when things are getting rolling. At the end of the beta, do something big and memorable to celebrate and thank all your beta testers.

The Key to Running a Good Beta is Two-Fold

Firstly, use real people to find flaws, do debugging and usage tests for you for free, while stress-testing critical systems in a fault-tolerant atmosphere. Secondly, use the beta time to start building the community; because the community that comes with the website is as much of a product as the service itself is. Missing one is like missing the paint on a car. Beta testers are an excellent resource for any starting community.

(Visited 22 times, 1 visits today)