How to maximise results with A/B testing

When reviewing your campaign, you may find that you’ve got multiple pieces of digital content you’d like to test. And it can be tempting to test them all simultaneously to save on overall costs. But this is where organisations often fall down with A/B testing as testing multiple variants at the same time can really confuse results. To make your results as clear as possible, you should test just one variable at a time.

The same goes for your measurables. Whilst you’ll have lots of metrics you’ll be looking to track in each A/B test, you should choose one primary goal metric to focus on before you begin the test. This will help you to stay focused and will make sure that you set up the test in the most effective way for measuring against your chosen goal.

You should also determine the sample size of your audience, and this will depend on which variable you’re testing. For example, if you’re testing an email then you’ll need to send an A/B test to a smaller group to get more statistically meaningful results.

If you’re looking to run an A/B test on your website or for an email, you’ll need to enlist the help of an A/B testing tool. If you’re a HubSpot user (like we are) then you’ve got various A/B testing options to choose from, including A/B testing tools for email, calls-to-action and landing pages. If you’re not a HubSpot user then Google Analytics Experiments is also a great option and it’s even free to use.

Timing is everything when it comes to A/B testing, so make sure that the test runs for long enough to obtain useful data. The duration of your test will essentially depend on how much traffic you get – so if you get a small volume of website traffic then your test will take longer.

Whilst quantitative data will play a key role in your A/B test, it won’t answer the question of ‘why’ your audience prefers certain features over others. So whilst you’re running your test, it’s a good idea to collect qualitative feedback from your audience. This can be achieved through a survey or poll that can be generated as part of the overall test.

Once your test is complete, you’ll need to focus once again on that primary goal metric when it comes to measuring and analysing your results. There’s always a risk of getting swept up in other metrics here, which may mislead you to which has ultimately performed better for you. For example, if you chose conversion rates on an online form, don’t get distracted by the open rates if that version of the form didn’t have a high conversion.

Interpreting the data from your A/B test can be tricky, so if you’re not comfortable doing this manually you can always put the results from your test into HubSpot’s free A/B testing calculator.

Once you’ve interpreted your results and determined a clear winner, you can use the results to shape how you proceed with your campaign.

If you’re looking to run an A/B test and would like some support with planning, running or measuring it (or all three), get in touch and we’d love to help.