I recently spoke about A/B Testing on Merita Business Podcast. However it was conducted in Italian so I decided to translate my answers here:
What is an A/B Test?
The A/B Test is one fo the many digital marketer’s tools. When you want to assess the impact of a change on your website, you can use a split test to isolate your change from other independent agents.
Why should we take A/B Test in consideration on our online businesses?
Because the alternative is to trust blindly your gut feeling and in many cases, it’s just plain wrong. A split test helps us to validate our right hypotheses and discard incorrect assumptions, helping us to keep our direction towards success.
Is this applicable to sites with little traffic, let’s say in the 100-visit per day?
Split testing does not admin shortcuts. There is a minimum amount of traffic, to conduct significant tests. Conversion rates are also important to consider. A little traffic or a small conversion rate can take us to the distorted field of insignificant tests, and we should stay away from there. There online calculators to determine the minimum amount of traffic to conduct a significant test. We should strictly follow them all the times.
We should also make sure we run tests for a reasonable amount of time. Usually, tests should last longer than a business cycle, so to make sure complete flows are tested. We should also try to overcome seasonalities. As a rule fo thumb, a test should never run shorter than a full week.
What can I A/B Test on a given site?
Technically we can test everything, in one way or another. However not everything makes sense to be A/B tested. The best tests are the ones with few clear visual elements and direct consequences. For instance: “is it better to have the checkout button on the right or the left of the page?”, “is it better to have five questions on the same page or 5 pages with one question each?”, or “do we lose conversions if we add a mandatory field in the checkout form?”.
How do you select and prioritise the elements to tests?
First create a list of all the actionable items on your website. Then add, next to every item, the amount of work necessary to create alternative variations. Next to it add the potential gain you expect from every element. Then just sort the list by the maximum gain at the minimum cost and start iterating.
Which are the best tools to start with A/B testing?
Optimizely is the tool to start with. It offers a great visual editor and it just requires a minimal implementation. There are easy plugins for WordPress that will abilitate Opmizely in a matter of a few clicks. We can also mention Unbounce, Kissmetrics and Google Analytics Content Experiment.
Can you tell us about unexpected results from A/B Tests you conducted recently?
On our platform, we run several A/B tests in parallel at any given time. And almost every day we have unexpected results. And that is what we are after: unexpected results. The practice of A/B testing requires patience and the ability to accept errors as default outcome of a test. Identifying errors and fixing them is our job and even if being wrong nine times out of ten, it pays well in the long run.
Leave a Reply