Last Updated on
Whether you have a website, apps, landing pages, or are focused on omnichannel and personalization, A/B testing is a proven method to boost your conversion rates.
Even small changes can yield double-digit percentage increases in engagement, average order size, in-store visits, and sales.
What is A/B Testing?
Behind most A/B testing is a simple concept: you choose an element on a web page or landing page to alter.
Then you test and see if visitor behavior differs between the test page and the control (original) page. That visitor behavior might include click-throughs, purchases, sign-ups, booking appointments, and placing items on wish lists.
Elements that are commonly tested include:
- Call-to-action text
- Call-to-action buttons (shape, color, size, placement)
- Visual design
- Shopping cart buttons (shape, color, size, placement)
When testing includes personalization, then a visitor data point (like geolocation) is used to display personalized content to each visitor (like sales at a location near them). Then visitor behavior on the new page is compared with the old page that didn’t take geolocation into account.
It’s not uncommon for businesses to be running hundreds of tests on any given day.
Other Types of Testing
A/B testing (sometimes referred to as split testing) is the type that gets the most attention, but there are other types of tests that you can run to get information on how your visitors behave.
Multivariate testing is the testing of change combinations to see their impact. Whereas A/B testing keeps things simple by comparing just one change, multivariate testing allows you to test two or more changes to measure the effectiveness of the combination as a whole.
Depending on the granularity of the data you collect, you may be able to identify the impact of individual components (but this is not guaranteed).
These types of testing systems boost conversions by using geolocation and online behavior to show relevant content to each individual.
Heatmaps are another way of gathering information about users. They are visual displays that use color to indicate where visitors click the most and how far down they scroll on a page.
Heatmaps can also display the results of eye-tracking tests, revealing which parts of a page visitors focus on the most, and which parts are largely ignored.
A/B Testing Metrics
Before beginning an A/B testing campaign you’ll need to decide what you are going to test and measure.
What you measure impacts the outcome and success of your experiments. Generally speaking, the more specific your metric, the more actionable your results will likely be.
Some typically used metrics include conversion rate, call-to-action effectiveness, and whether the visitor clicks on the pop-ups shown.
If the data is strong enough for you to draw conclusions, you will know whether you should change your site and what you should change.
With such data, you’ll be able to improve user experience, personalization, and targeting so that your website becomes an effective tool for promoting your business.
Which A/B Testing Tool is Best For My Business?
To narrow down your choices, start with your purpose. For those who are in a rush, we’ve made some recommendations below to get you started in your search.
- If you need A/B testing for landing pages, check out Unbounce or Leadformly
- If you need A/B testing for your website and apps, explore VWO, Optimizely, or Convert.
- If you need to optimize personalization across channels look into Qubit or Adobe Target.
The Top A/B Testing Tools
A/B testing is so commonly done that there are a variety of tools designed to simplify the process of setting up tests, collecting data, and analyzing the results.
The best option for you depends on your website and what you’re testing, but we hope our summaries of the options currently available help you curate a list for further research.
Optimizely is a full platform that allows you to experiment with, test, and deploy website changes.
You can run tests on all aspects of your website. Optimizely’s built-in sharing tools allow you to include multiple parties when it comes to your testing projects.
Convert is a tool designed to help with A/B testing and website personalization.
It is fully compliant with GDPR since it stores no personal data by default and only uses first-party cookies for accurate tracking.
Convert also boasts that they’re one of the fastest testing tools, which means that you don’t have to worry about your test slowing down your website.
VWO is a testing suite that allows you to integrate A/B testing into your business process without requiring any coding or IT involvement.
VWO features session recording, heatmaps, surveys, and more to help you gather information on how visitors are responding to your website pages.
VWO also allows you to test across channels, so you can validate things like your search algorithms and subscription workflows.
Optimize is Google’s A/B testing suite built on the back of its Analytics product.
You provide the metric you want to measure and Optimize allows you to run tests to see how well they perform.
Optimize supports A/B tests, as well as redirect tests (a type of A/B test that pits separate web pages against one another) and multivariate tests (which tests variants of 2+ elements at once to see which combination is best).
Strictly speaking, HubSpot isn’t a testing suite, but it’s customer relationship management (CRM) suite include testing tools.
With HubSpot, you can test your contact-based workflow to see how well your enrollment call-to-actions are working, A/B testing on your website pages, and your marketing emails.
Omniconvert offers two primary products: an optimization tool that you can use to increase conversion rates and professional services designed to achieve the same goal.
Omniconvert begins by focusing on five metrics: number of visits per month, number of page views per month, conversion rate, average order value, and margin.
With that information in mind, Omniconvert helps you develop a plan for improving your bottom line.
Apptimize is a cross-platform (e.g., mobile apps, mobile sites, websites, in-store, and so on), all-in-one A/B testing and release management suite.
While Apptimize has always been strong at assisting improvements to mobile experiences, the company has been increasingly focused on other areas where you might have contact with your customers.
Crazy Egg helps you learn more about your customer’s experience with your website via snapshots, heatmaps, and session recordings so you can see exactly how people interact with your website.
You can also create and implement AB tests. The included editor makes it easy for you to make quick changes for testing purposes, then push the changes out to your production site.
AB Tasty is a testing platform that comes with features to make the testing process easier, such as an easy-to-use editor that allows for changes without coding.
AB Tasty’s tests rely on session recordings and heatmaps to gather data. It then uses artificial intelligence (AI) to help you draw conclusions from the gathered data.
Leadformly is a niche product that offers A/B testing strictly for forms. If you want to improve your forms so that you gather the most information possible without causing your visitors to quit halfway through, this product is for you.
Leadformly comes with optimizes templates, interactive questions you can use, and lead segmentation, conditional logic, and intelligence validation so that you can build the correct forms for testing and use.
Those who want a simple, easy-to-use tool to design and build effective landing pages should look into Unbounce.
In addition to providing you with the design and development tools you need, Unbounce comes with A/B testing so that you know that the changes you are implementing are the right ones for you.
If you are a WordPress user who wants to make sure that your posts and pages are optimal, consider the Nelio AB Testing plugin.
This plugin features an easy-to-use interface built into your WordPress dashboard that helps you create, manage, and track all of your A/B tests.
You’ll also be able to gather data via heatmaps. Note that this plugin also works with WooCommerce.
Adobe Target is an A/B and multivariate testing solution so that you can test both individual changes and change combinations to find the solution that is right for your website.
Furthermore, Adobe Target is an omnichannel solution so that you can optimize your presence everywhere.
Adobe Target’s AI means that your tests are appropriately personalized and effectively administered to your visitors.
This video provides an easy-to-understand overview of how Adobe Target can provide effective personalization.
Oracle’s Maxymiser is a suite of tools to help you test your website and implement personalizations.
Maxymiser can handle anything from simple A/B tests to complex multivariate tests across multiple devices and sites.
Results are available to you via its reporting and analytics tools.
SiteSpect is a tool that supports both client and server-side A/B testing.
You can leverage the tool on your own, but you can also partner with the SiteSpect team. SiteSpect maintains that all of your testing data will remain secure — the company is PCI-DSS certified and GDPR- and HIPAA-compliant.
In addition to its testing tools, SiteSpect offers personalization assistance and website optimization so that your pages load as quickly as possible.
Monetate is a platform aimed at helping e-commerce brands with things like discovery, lead segmentation, A/B testing, and website optimization.
Monetate supports multi-platform and multi-channel testing and helps with proper implementation of your website personalization behavior.
Qubit’s primary purpose is to assist with personalization. Its software enables you to add rich layers of context and relevance to what your users see based on data you’ve collected.
You can then run tests and optimize based on the results of your tests.
Visit the Qubit website or YouTube channel to hear the interesting case studies of how Qubit has been used to increase website engagement, in-store visits, conversion rate, ROI, and average order value.
Video: Find out how Qubit enabled Indochino to increase revenue by $1 million in their first year of using the analytics platform. [Indochino also uses Unbounce landing pages in conjunction with Facebook Ads. With Unbounce’s A/B testing, Indochino has achieved conversion rates between 17% – 30%. (Source: Unbounce YouTube channel video)]
Evolv is a platform that allows you to test multiple ideas simultaneously so that you can find the change combinations that improve your website the most.
Furthermore, Evolv claims that its machine learning, AI-based tools can provide you with the data you need after just a month, rather than years, and its “evolutionary algorithms” can help you develop the best data-driven web design possible.
Google’s popular Analytics tool comes with an experiments framework that allows you to test changes or variations on your websites to see its impact on your goals.
You can decide on the changes to test, or you can let Analytics make the optimization decisions for the experiment on your behalf.
The analytics framework allows you to test both client- and server-side changes.
Popular Tools to Use with A/B Testing
In addition to A/B testing suites, which typically help with things like experimental design and setup, data collection, and reporting/analytics, other tools can streamline and improve your testing processes.
- UsabilityHub helps you determine the best design decisions to test
- Google Analytics is a robust tracking software that can help you gather the data you need
- Kissmetrics is a data collection suite that you can think of as a more advanced version of Google Analytics
- User Testing is a way to streamline the process of working with real users to gather preliminary data that can drive the design of your experiments
- Clicktale offers machine learning-based/AI-driven solutions that can help you analyze data you collect
- HotJar offers tools you can use to collect data about your users’ behaviors, including heatmaps, session recordings, bounce rate tracking, and more
- Inspectlet helps you create session recordings that you can review to see every mouse movement, scroll, click, and more that your users do
Here are a couple of common questions we receive from readers. Have a question of your own? Leave it in the comments section below and we’d be delighted to answer it.
What is A/B testing in SEO?
Unlike A/B testing focused on conversion rate optimization (CRO), SEO testing entails testing groups of pages. You select a control group and a set of testing pages.
The test pages don’t have duplicate content but are generally similar to pages in the control group. For example, a hiking website might use a day hike in a particular park as a control page and a day hike in a different park as a testing page; plus a hut-to-hut hike on one mountain as a control page and a hut-to-hut hike on a different mountain as a testing page.
On the test pages, SEO optimizations are introduced that don’t exist on the control pages.
Does A/B testing affect SEO?
No, A/B testing does not affect SEO. Google confirmed this years ago on their Webmasters blog when they stated that neither A/B or multivariate testing affects SEO.
However, they do suggest that misuse of A/B testing, such as “cloaking,” could affect SEO.
If you’re testing with multiple URLs they recommend using the rel=”canonical” link attribute to signal the identity of the control page.
When you create or update your website, it’s important to know what changes to make that have the greatest positive impact on your customers. And, with today’s data gathering and testing tools, you don’t have to make any guesses.
There are a variety of tools on the market designed to help you make informed decisions that will improve your users’ experience with your website, increase conversion rates, and improve your bottom line.
The best tools for you depends on your goals and the types of channels you are using to reach your audience, but in this article, we covered a variety of options that should fit the needs of most users.