A/B testing in email marketing - what is it and why should you do it?

When embarking on an email campaign, analysing what’s working is crucial for maximising return on investment. When you know what’s working (and what isn’t), you know what to do more (or less) of. By incorporating different calls to action or subject lines in your email and sending different versions to a small section of your data list, you can determine which one attracts more interaction and implement the most popular into your main send. This type of testing is known as split-testing or a/b testing.


Marketing decisions are too important to rely on ‘gut feel’. When your CMO asks why a particular campaign has bombed, you want to be able to say you had more than intuition to base your strategy on; you want evidence. A/B testing provides that evidence.


So, how does it work? Start by picking an element you want to test. Some examples are:


  • Subject line – (make one a question and one a statement, or change the tone from serious to humorous) 
  • Button label – try Buy Now and More Info to see which generates the most clicks
  • Button colour – do people click more on the Buy Now button in green or the Buy Now button in red?
  • Button size – bigger or smaller?
  • Email content – does the video link generate clicks, or does the blog link?


You get the idea. 


Decide what your goal is, i.e. your measure of success. You might want to track open rates, click-through rates (CTR), and actual sales – they’re pretty standard metrics for determining the success of a campaign. Pick the one element you want to test, select a randomised portion of your data list, and dispatch the email. Allow enough time to get a true reading of the results; there’s no point picking up the analytics twenty minutes after the email has been sent; you have to give your audience time to see the email. After all, people may be at work, dropping the kids off at school, or just generally in the middle of something, and it isn’t convenient to look at an email the moment it lands.




Measuring performance is a crucial step in a/b testing; after all, what’s the point in testing if you don’t look at the results?


Trawling through the figures, you should be able to determine which test has produced the most interactions. If you have Email-Heatmaps, you can overlay the heatmap comparing the two campaigns to see which worked better; the ‘hottest’ one is the winner. 


There are other factors to consider when interpreting results: the open rate may be high, but if the click-through rate is low, then it may be worth split-testing the call-to-action. If the CTR is high but the conversion is low, take a hard look at what you’re offering and split test elements of that to see where the barrier lies. The other thing to consider is what device each worked best on. For example, it could be the bigger button worked better for users who interacted with the email on a desktop computer, and the smaller button generated more interaction for mobile users. If you have tagged your data list to include device preference, you can include this data in your decision-making. 


In cases such as button colour and size testing, these findings can be implemented across future campaigns, although it’s generally a good plan to test these things more than once to get a good idea of mean success. For more changeable features, such as subject lines, whilst these will, out of necessity, change with each email to reflect the purpose of each campaign, you can infer whether questions or statements work best for your audience, or humour or the fear of missing out factor generates more opens. Then consider if that translates into the end goal, i.e. buy now.


The real key with a/b testing is only to test one element at a time so you get accurate measurements of which change worked. Whatever the figures show, you can implement, so a/b testing is the way to provide evidence of a logical approach to email marketing. Try it today!

Red Line
Yellow Stroke Circle

Start your free 14-day trial in just3 easy steps

Mauris sit amet ante odio. Ut lorem nisl, porta nec nulla vel semper
fringilla diam. Quisque varius ac ligula quis iaculis.
Step Img1
Step 1 Tag
Choose your plan

Select your plan that is based on total campaigns sent in your email platform you wish to report on. (not monthly sends – we pull all your sent campaigns but you select how many you wish to view)

Step Img2
Step 2 Tag
Create an account

Enter your details, credit card for a 14-day free trial (no notice period and cancel at any time!)

Step Img3
Step 3 Tag
Add your Integration

Simply connect EmailHeatmaps to your email marketing platform via API key or Mailchimp direct login.