A Guide to Split Testing

What is A/B split testing

A/B split testing is a variety of marketing testing where customers randomly receive almost identical, yet slightly different, test samples. This method can efficiently be used in various niches such as brochures, email campaigns and landing pages.

Its exactly what it sounds like: you have two versions of an element (A and B) and a metric that defines success. To determine which version is better, you subject both versions to experimentation simultaneously. In the end, you measure which version was more successful and select that version for real-world use.

A/B testing is still not as common like other Internet marketing subjects as SEO, Web analytics and usability. People don’t completely understand what it is or how it could benefit them or how they should use it. Split testing can help you to improve conversion rates, better understand visitor behavior, test out new products and much more.

Little change = Big difference

Testing can improve a site’s performance, usability and conversions. What’s more, it costs you comparatively nothing. Some examples include;

Two Magical Words Increased Conversion Rate by 28%

The words “It’s free” increased the clicks on this sign-up button by 28%, illustrating the importance of testing call-to-action buttons and how minor changes can have surprisingly major results.

Writing Decisions: Headline Tests on the Highrise Sign-Up Page

37signals tested the headline on its pricing page. It found that “30-Day Free Trial on All Accounts” generated 30% more sign-ups than the original “Start a Highrise Account”.

Human Photos Double Conversion Rates

A surprising conclusion from two separate A/B tests: putting human photos on a website increases conversion rates by as much as double. Scientific research backs this up, saying that we are subconsciously attracted to images with people.

“You Should Follow Me on Twitter Here”

This much-hyped split-test involved testing multiple versions of a call to action for Twitter followers. The study found that “You should follow me on Twitter here” worked 173% better than the control text, “I’m on Twitter.”

A complete redesign of product page increased sales by 20%

A software product company redesigned their product page to give it a modern look and added trust building elements (such as seals, guarantees, etc.). End result: they managed to increase total sales by 20%.

Popular tools for A/B Split Testing

Testing tools are available online for A/B testing, with different focuses, price points and feature sets. The following are some:

a. Google Website Optimizer

A free A/B testing tool from the search giant. A great option to get started, but lacks advanced features.

b. A/Bingo and Vanity

Server-side frameworks for Ruby on Rails developers. Requires programming and integration in code.

c. Vertster, SiteSpect, Webtrends Optimize and Omniture’s Test&Target

Enterprise testing tools

d. Visual Website Optimizer

An easy-to-use A/B testing tool, with advanced features such as WYSIWYG editor, click maps, visitor segmentation and tag-less integration. (Disclaimer: my start-up.)

e. Unbounce and Performable

Landing-page creators with integrated A/B testing.

How to do a split test

Split testing is where the money is. And it’s easy if you have a process to follow, you literally just do it, step by step, and in most cases, with little effort – you’ll be making more money in no time.

Even though A/B testing is super-simple in concept, keep some practical things in mind. we’re going to show you a real easy 5 step split testing process that almost anyone can follow anytime.

Step 1: Make Your Four Versions

Take whatever product you’re currently offering right now – and create a second version of the advertisement to split test against it. Here’s the only tweak you’re going to make for this version – change the offer from a “one time payment” to a “trial offer”.

Example – If you’re selling a product for $47 on weight loss… change it. Instead of $47 up front, they pay $2.95 to try out the product for 7 days. If they don’t cancel their trial within 7 days then at the end of 7 days they are billed $47 dollars. The next page you’re going to create is almost exactly the same – the only difference is the trial period. It’s now $2.95 for the first 14 days then $47 after the trial is up. The last “version” you’ll create is where we change the price. It’s $2.95 for 7 days, then if they don’t cancel they’ll be automatically billed $67.

That’s it. You’ll have four versions when you’re done:

1. Your original advertisement
2. A $2.95 trial with balance due in 7 days
3. A $2.95 trial with balance due in 14 days
4. A $2.95 trial with a PRICE INCREASED balance due in 7 days

Step 2 – Set Up the Split Test

If you already have some great split testing software and know how to use it – then use it. If not, here is a free solution for you that works just about as good as any.

It’s Google Website Optimizer – https://www.google.com/analytics/siteopt/splash?hl=en

It’s easy to use. Pick A/B split-testing. Then put in your original advertisement page URL and the URL someone would be forwarded to if they purchased your product or service (called your conversion page). Then put in your three experiment page URLs (the three split testing versions you created in step 1). When you’re done they’ll give you further instructions on how to use a simple code so that way Google can track it for you.

Step 3 – Analyze Results

Google Optimizer is specifically recommend simply because it’s slim. Meaning there are not a ton of metrics – just conversion rates.

It’s actually way too easy to get distracted by a ton of data and be more confused about what to do than before you started split testing. Wait first until each page gets at least 100 unique visitors. Ideally, you’d like at least 300 unique visitors, but if you don’t get a lot of traffic then 100 is “good enough”.

It won’t be as statistically valid but it will give you a good “ball park” figure. Once you get at least 100 unique visitors per page (or ideally 300) then you simply look at your results. One of them will be better than the others. Simple as that.

Step 4 – Headlines

Let’s say the winner was the $2.95 trial for 7 days offer. So what you’d do is take this page, and create 3 new pages to split test against the original version.

This time you’re going to test it against a few alternative headlines. The first headline should mention the fact that you are offering this for a $2.95 trial. The second headline should be an “opposite length” headline.

If the headline on your winning split test page was long, try it against a short headline. For example, if you were selling golf product and your original winner had a short headline that said, “Add 25+ Yards to Your Swing Before Your Next Round of Golf.” Then test it against another powerful headline.

Or vice versa. If the original winner had a “middle length” headline – neither very long or very short – then create versions so you’ll be testing a long headline, a short headline and a medium headline that all have pretty much the same hook.

Always test a simple, straight forward “how to” instant gratification headline. This is a headline that starts with “how to” and promises a big benefit in a short, specific period of TIME.

When you’re done you have these headlines to test:

1. Your regular headline
2. Your regular headline with mention of the TRIAL OFFER (if a trial offers one)
3. A real short headline
4. A real long headline
5. An “average length” headline
6. A “how to” instant gratification headline Roll it out and see which one wins. Then move on to the last step.

Step 5 – Long Vs. Short

Now we take your winner and put it through one more test. We want to test length. Take your sales letter “as is” and split test it against a “shorter, more condensed” version and a “longer, more thorough” version.

Let’s say your sales letter is 4 pages long. Try to cut it down to 2 pages. Remove anything you feel is not absolutely required. That’s one version to test. The other is to ADD to it. This usually means explaining additional benefits they get from your product that you didn’t mention before. It also means adding in more proof, if you can, and also spending more time explaining why what you’re offering is a good deal. It also means adding more testimonials if you have them.

You have a short, medium and long sales letter to test. So test them and see what happens.