August 3, 2017

User Research: What A/B testing is and how it killed personal opinions

You are launching a new marketing campaign but you struggle. It’s difficult to choose which visual, which claim works best from all those that you like. What usually happens is that in a committee you start a conversation about that. The conversation often transforms itself into a heated debate. Everyone has another idea of the impact each option could have. After hours and hours of unfruitful debates, you finally vote. You select one key visual for the campaign. Ouff! That was hard. Couldn’t that be easier? Of course, you know which is our answer. Yes, it can be pretty simple.

A/B Testing

Preference versus action

When committees decide which look the campaign should have you have a problem. It’s a problem because people base their decision on personal preference. They don’t take into account the preferences of their customers. It is not based on data that shows what makes the targeted people, your users or customers, act. You could imagine what your customers like. That’s good. But in fact, we are more interested in what really makes your customer act. If they don’t like an ad, but still click on it, it’s better than a beautiful ad which leads to no click.

presenting from a different perspective
works for presenting all kinds of information

What is A/B testing?

A/B testing, or in general split testing, is a method that helps to beat opinion with data. Instead of choosing by yourself which version or variation you should use you test them out.

the audience tells you what they want
results are used to tailor to the audience

How to run an A/B testing

The first step is to setup a testing group. To do this you select a sample of your target group. Then you divide your target group by the number of variations you want to test. If we are talking about an A/B testing you have two variations. If you have more than two variations then the technical term is “split testing”.

The second phase is to let people test it out. Let’s say we do an A/B testing of online ads. You want to test out what makes people go to your website. Is it the wonderful price of your product or its features? In your test, you have one ad with the price written in big, and one ad with the features or product benefit explained.  The first half of your possible customers sees the ad with the prices. The second half sees the ad with the product benefits.

The third phase is controlling. To make it simple, the ad that generates the more clicks on your websites is the winner. In reality, this is a bit more complex. There are more parameters that go in the optimization of online ads, but that’s for another article.

looking at each aspect of the profile
analyzing aspects of each type of audience

What can we A/B test?

We can use A/B testing as a method to take out a lot of discussions from the meeting room. With A/B testing you can get data that shows you which direction to take. Here a few examples of what you can find out with a good A/B testing strategy:


  • Which color should be your calls to action?

  • What copy creates more conversation?

  • What works better for women and what works better for men?

  • What works better in each region/culture?

  • What price are your customers willing to pay for your product?

we can find out how your audience wants to be convinced
we improved conversion by 500% for Ragusa

A/B testing: up to 500% more clicks

Every advertising campaign that Enigma launches includes A/B testing. We don’t stop there, we use also A/B testing to optimize websites. We use it to create innovative services and products.

One example is particularly striking:

Through consistent A/B testing, the performance of a conversion campaign for the Chocolate brand Ragusa For Friends could be improved by over 500% compared to the first version made without the support of Enigma.

optimize your campaigns before they launch
don’t miss the opportunity to test

A campaign without A/B testing is dumb

When you see the results that we highlighted before the conclusion is clear. Running a marketing campaign without using A/B testing is a missed opportunity. It’s like throwing your media space money out of the window. With a good A/B testing strategy, you can optimize your campaigns. You can optimize them so that without changing the media space you get more results and even lower the costs per result.

Discover more about the power of A/B testing

In the following weeks, we will publish more articles about the power of A/B testing. To get notified of their publication, join our newsletter using the form below.

I want to thank Patrizia Lamprecht for her help in this article. With her experience in A/B testing, she was a great support.

Stay tuned
Don't miss Enigma’s latest updates. News, projects, blog articles & inventions.
360° agency
Tailor-made strategies