No matter what business you’re in, you’re putting words out there. 

Website content. Ads. Social media posts. Emails. Pop-ups. Call to action.

There’s a ton of information that you’re sharing with your audience every day! 

But how often do you test the effectiveness of what you’re saying? 

And if you aren’t testing… How do you know what’s working and what isn’t? 

Learning how to do A/B testing could be the biggest skill you learn this year! 

Here’s everything you need to know about testing your campaigns for effectiveness. 

What is A/B testing?

A/B testing (also called split testing) is a simple comparison test between two similar versions of the same thing. 

The point is to see which one performs better. 

This can give you interesting and important insight into your audience’s mindset! 

Analysing how people respond to certain content will help you to make more effective decisions about your own marketing campaigns.

How Does A/B Testing Work?

A/B testing is a simple principle. 

You take a particular digital asset… 

And create two slightly different versions of it. 

The key is to only change a single variable. 

Maybe it’s a personalised greeting vs a generic greeting. 

Maybe it’s a more direct CTA vs a less direct one. 

It could be a bit more humour in the content vs more formal content. 

Or even something like changing the colour of your CTA button. 

And then see how people respond. 

You can change multiple variables if you want… 

But it becomes more difficult to keep track of which change really makes a difference with your audience. 

Most platforms—website, email, social media—come with built-in split testing tools. 

So don’t worry if comparing two assets seems complicated! 

It’s easy when you know how to set it up and analyse the results (which we’ll discuss later). 

Why Should You Do A/B Testing?

How will you ever know what works best for your audience if you don’t test it? 

A/B testing can provide insights into how customers feel about your product, service, and customer experience. 

Which means you can use this info to tweak things that aren’t quite working… 

To improve your customers’ experience. 

And an improved customer experience means more sales for you. 

Possibly more referrals. 

And definitely happier customers! 

It’s a win-win situation for you and them. 

• Not sure if your pop-up is catching attention? A/B test it. 

• Wondering if you could make more sales with your welcome email? A/B test it. 

• Not totally sure if your offer works for your target market? A/B test that too. 

If you find that certain things don’t get a great response, you know what not to do. 

And you can try to replicate the things that do get a good reaction. 

What Can You A/B Test?

Once you know how to do A/B testing, you can test almost any digital resource you have. 

You can get very detailed in terms of testing marketing messages, design, colours, and almost every variable that goes into designing resources. 

• Pop-ups 
• Offers 
• Landing pages 
• Emails 
• Lead magnets 
• Page layouts 
• Colours 
• Social media posts 
• CTA text & delivery methods 
• Photos and product images 
• Keyword volume 
• Push notifications 

When Should You Use A/B Testing?

One of the great things about A/B testing is that you can do it whenever you feel the need. 

If you notice that your latest pop-up isn’t getting the results you expected, an A/B test can give some insight into why. 

Similarly, if your welcome email/landing pages/social media posts aren’t converting well, an A/B test could determine the cause. 

You can do random A/B testing throughout the year whenever you feel like you want a bit of insight into what’s going on in your customers’ minds. 

That being said, rather than just randomly testing stuff, it’s a good idea to use your analytics to get a clear idea of what to test and why you want to test it. 

Using analytics, you can see things like: 

• Your most popular pages 
• Where traffic arrives from 
• How long readers spend on a page 
• Which pages have high bounce rates 
• What readers click on 
• How long viewers pause and where 

This will help you decide where to focus your A/B tests to get the greatest ROI when it comes to finding our info on what your customers want. 

It’s also a great idea to make use of user tests, which allow you to watch readers in real-time to see how they navigate your website and where they get stuck on things. 

If you need more direct information, you can always send out a survey. This way, you’ll get direct feedback from your customers on their pain points. 

Remember, customers’ wants and needs change over time. 

It’s in your best interest to perform research on a regular basis so you know you’re always providing practical, easy solutions! 

How Long Should A/B Tests Run?

To get accurate numbers, you’ll need to get a certain number of traffic to the asset. 

You can’t get accurate results if only 100 people test it! 

Most platforms come with built-in A/B testing software that show you when the test has reached the point where it’s considered “statistically significant”. 

Now, most marketers will end the test here. 

It means you have enough data to make reasonable assumptions about your customers’ behaviour and adjust your assets based on that. 

But it’s not the most accurate the test can get. 

If you don’t have a lot of time, you can stop when you reach this stage. 

But if you want proper insight into your customers’ behaviour… 

As well as what drives certain behaviours… 

It’s best to run your A/B tests through one full business cycle at least. 

If not two. 

This allows you insight into how customer behaviour changes based on what the economy is doing at the time. 

Benefits of A/B Testing

Unsure about doing A/B testing? 

Here are some benefits you can expect. 

• Better understanding of your audience 
• Keeping up with trends 
• Improved customer experience 
• Informed marketing choices 
• Lower bounce rates 
• Better conversion rates 
• Increased ROI 

How to Conduct A/B Testing

Want to try A/B testing on one of your resources? Here’s a quick checklist of how to do it effectively and with as little stress as possible. 

Choose Your Resource to Test 

Is your pop-up not converting well? 

Maybe you’re trying to make more sales with your welcome email. 

You can choose whatever resource you want to test. 

I advise sticking to one at a time, at least in the beginning. 

That will prevent information overload! 

Set a Goal 

Don’t just A/B test for the sake of it. 

Set a goal for your test, otherwise it’s hard to tell if it’s successful or not! 

For example, are you testing to see which pop-up gets more emails? 

Or which web page design has the longest time spent on it? 

Set your goal so you know what you’re looking for when it comes to analysing results. 

Choose One Variable to Change 

Don’t be tempted to change everything. 

Rather, choose one thing to test at first. 

Analyse your resource and try to figure out what’s causing it to convert poorly. 

It could be: 

• The design 
• The colours 
• Your message/language 
• Your CTA (style or text) 

Aside from being much easier, choosing just one variable has another practical advantage. 

It means you know exactly which variable is the problem. 

If you change the colour and you still don’t see results, you know it’s not the colour causing the problem. 

But if you change the colour, the design, and the CTA and then you see results… 

You’ll never know what the actual issue was. 

Once you’re good at single-variable A/B tests, you can try multivariate ones. 

Change Your Variable 

Once you’ve chosen your variable to test, create two separate versions of the resource. 

The only change will be your chosen variable. 

They should look almost like carbon copies, just with one specific change. 

The original one is known as the “control”. 

The one with the change is called the “challenger”. 

Create Your Splits 

The next thing to do is split your audience into two. 

This should be done randomly and equally. 

For example, let’s assume you’re testing an email on your list of 5,000. 

Split it straight down the middle: 2,500 and 2,500. 

If you’re testing another resource and using an A/B testing tool, some of them automatically split your traffic for you. 

You should also make sure this split is completely random. 

No choosing your most engaged email subscribers for one split and quiet ones for the other! 

This will give you the most honest and accurate results. 

Determine Sample Size 

If you’re testing an email, sample size is easy. 

But what if you’re testing something like a web page? 

There’s no way to determine how much traffic might land up there on any given day. 

In cases like these, you’ll need to run the test for long enough to get results that are “statistically significant”. 

You can’t get super accurate results on a sample size of 500 people. 

But if you get 5,000 visitors… 

That’s got a lot more weight behind it. 

Run One Test At a Time 

Although it may seem like the slow way of doing it, stick to one test at a time. 

Don’t test an email… 

And your pop-up… 

And a new web page design at the same time. 

Basically, the simpler the better. 

Do one test at a time so there’s no over-complicating things. 

How to Analyse A/B Test Results 

There’s no such thing as winning or losing an A/B test. 

Every result is significant. 

Keep an eye on your results on the dashboard of whatever tool you’re using. 

You need to understand that your A/B testing tool won’t do the analysis for you. 

It’s up to you to scan through your results and gani insight! 

Remember your original goal when you analyse your results. 

Were you aiming to find out which pop-up converted the best? 

Then don’t waste time looking at any other metric. 

Study the patterns and find your answer. 

If the answer isn’t what you expected, don’t be upset or worried. 

It’s valuable insight into what works and what doesn’t. 

Don’t be afraid to make changes based on your analysis! 

That’s the whole point of an A/B test. 

Final Thoughts: How to Do A/B Testing 

Learning how to A/B testing could be one of the best things you learn for your business. 

It’s easy and can even be fun! 

But it does require some patience and a discerning eye. 

Why not start small and A/b test something on your website today? 

You may be shocked at the results… 

But it could make positive changes in your business.