How do you use A/B testing to improve landing page conversions?

To use A/B testing to improve landing page conversions, create two versions of a page that differ by a single element, drive equal traffic to both, and measure which version produces more of your desired action. The key is testing one change at a time so you can attribute any improvement directly to that element. The questions below cover everything from where to start and how much traffic you need, to writing strong hypotheses and knowing when to call a winner.

What elements of a landing page should you A/B test first?

Start with the elements that have the greatest influence on whether a visitor takes action: your headline, your call-to-action button, and your hero section. These three components are typically the first things a visitor sees and the last thing standing between them and a conversion. Testing them first gives you the fastest route to meaningful improvement.

A practical order to follow:

• Headline: It sets expectations and determines whether visitors stay or leave. A clearer or more benefit-focused headline can lift conversions significantly on its own.
• Call-to-action (CTA) text and button: Changing “Submit” to “Get my free demo” is a low-effort test with potentially high impact.
• Hero image or video: Visuals shape first impressions. Test whether a product image, a person, or no image at all performs better for your audience.
• Form length: Reducing the number of fields often increases form completions, especially for top-of-funnel offers.
• Social proof placement: Test whether moving testimonials or trust badges closer to the CTA improves confidence at the point of decision.

Avoid starting with minor cosmetic changes like font size or footer colour. These rarely move the needle and waste testing cycles you could spend on higher-impact elements.

How many visitors do you need before A/B test results are reliable?

As a general rule, you need at least 100 conversions per variation before drawing conclusions, and ideally you should run the test for a minimum of two full weeks regardless of traffic volume. This ensures your results account for natural variation in visitor behaviour across different days and times.

The exact number of visitors required depends on three factors: your current conversion rate, the size of improvement you are trying to detect, and the statistical confidence level you are aiming for. Most testing tools use 95% confidence as the standard threshold, meaning there is only a 5% chance the result is due to random chance.

If your landing page currently converts at 2% and you want to detect a 20% relative improvement (meaning a new rate of 2.4%), you will need several thousand visitors per variation to reach statistical significance. If your conversion rate is higher, say 10%, you will need fewer visitors to detect the same relative lift.

The practical takeaway: low-traffic pages are genuinely difficult to test. If your page receives fewer than a few hundred visits per month, consider running tests over a longer period rather than cutting them short and acting on unreliable data.

What’s the difference between A/B testing and multivariate testing on landing pages?

A/B testing compares two versions of a page that differ in one element. Multivariate testing simultaneously tests multiple elements and multiple variations of each, allowing you to see how different combinations of changes interact with each other. A/B testing is simpler and requires less traffic; multivariate testing is more powerful but demands significantly more visitors to reach reliable conclusions.

For example, an A/B test might compare a blue CTA button against a green one. A multivariate test on the same page might simultaneously test two button colours, three headline variations, and two hero images, generating many possible combinations for the testing engine to evaluate.

The right choice depends on your situation:

• Use A/B testing when you have a specific hypothesis about one element, when traffic is limited, or when you are early in your optimisation process.
• Use multivariate testing when you have high traffic, want to understand how elements interact, and need to optimise several components at once without running sequential tests over months.

For most teams, A/B testing is the right starting point. Multivariate testing becomes valuable once you have already made significant gains through A/B tests and are looking for incremental improvements across a well-established page.

How do you write a hypothesis for a landing page A/B test?

A strong A/B test hypothesis follows this structure: “Because [observation or insight], changing [specific element] to [specific variation] will [expected outcome] for [target audience].” This format forces you to ground your test in a real reason rather than guessing, which makes your results far more actionable whether the test wins or loses.

Here is an example of a weak hypothesis: “Let’s try a different headline.” Here is the same idea written as a proper hypothesis: “Because our heatmap data shows visitors are not scrolling past the hero section, changing our headline from a product description to a benefit-led statement will increase form submissions from first-time visitors.”

A well-formed hypothesis has three qualities:

1. It is based on evidence. Use data from analytics, heatmaps, session recordings, or user feedback rather than intuition alone.
2. It is falsifiable. You can clearly determine whether the outcome was achieved or not.
3. It isolates one variable. If you change multiple things, you cannot know which change caused the result.

Writing hypotheses this way also builds a testing log over time, which helps your team learn from patterns across multiple tests rather than treating each one in isolation.

Why do A/B tests on landing pages fail to produce results?

Most A/B tests fail to produce actionable results because they are ended too early, test elements that have too little impact on behaviour, or are based on weak hypotheses rather than genuine user insights. The result is a library of inconclusive tests that leave teams no wiser than when they started.

The most common reasons tests fail:

• Stopping too early: Ending a test as soon as one variation takes a lead, before reaching statistical significance, produces false positives. Results fluctuate naturally in the early days of a test.
• Testing trivial changes: Testing minor visual tweaks that visitors barely notice will rarely produce a detectable difference in conversion rate.
• Ignoring external factors: Running a test during an unusual period (a sale, a seasonal spike, a news event) can skew results in ways that do not reflect normal behaviour.
• Not segmenting results: A test that shows no overall winner might still reveal that one variation performs significantly better for a specific segment, such as mobile users or returning visitors.
• Polluted traffic: If your test page receives a mix of very different audiences (for example, paid traffic and organic traffic with very different intent), aggregate results can mask meaningful differences.

The fix is discipline: write a hypothesis before you start, set your sample size in advance, let the test run its course, and segment your results before concluding there is no winner.

How do you measure whether an A/B test actually improved conversions?

Measure the primary conversion metric you defined before the test began, typically form submissions, sign-ups, or purchases, and compare the conversion rate between the control and the variation once you have reached statistical significance. The percentage difference in conversion rate between the two versions, combined with your confidence level, tells you whether the improvement is real.

Beyond the headline conversion rate, consider these supporting metrics:

• Revenue per visitor: A higher conversion rate does not always mean more revenue if the variation attracts lower-quality leads.
• Bounce rate: If the variation reduces bounces but does not improve conversions, it may be improving engagement without resolving the core barrier.
• Time on page: Useful context, but not a conversion metric on its own.
• Micro-conversions: Clicks on the CTA, scroll depth, and video plays can help explain why one version outperformed the other.

The most important rule: define your success metric before you launch the test, not after. Choosing your metric based on whichever number looks best once results are in is a form of data manipulation that leads to poor decisions.

When should you stop an A/B test and declare a winner?

Stop an A/B test and declare a winner when you have reached your pre-set statistical significance threshold (typically 95% confidence), collected at least the minimum number of conversions per variation you calculated before starting, and run the test for at least two full weeks to account for weekly behavioural cycles. All three conditions should be met before you act on the result.

There are also valid reasons to stop a test early without a winner:

• The test has run for four or more weeks with no sign of significance, suggesting the change is unlikely to produce a meaningful difference.
• An external event has significantly disrupted traffic patterns or user behaviour.
• A technical issue has affected one variation’s performance.

When you do declare a winner, implement the winning variation and document what you learned. Even a losing variation teaches you something about your audience. The best optimisation programmes treat every test result, positive or negative, as an input into the next hypothesis.

How Spotler helps you test and personalise your landing pages

Running structured A/B tests is only half the picture. Once you know what works, the next step is delivering the right experience to the right visitor automatically. That is exactly what we built Spotler Website Personalisation to do.

With Spotler Website Personalisation, you can:

• Run built-in A/B tests to measure exactly which personalised content performs best per audience segment.
• Dynamically adapt page content based on company data, click behaviour, and stage in the customer journey, so visitors who arrive via an email campaign see content that matches that campaign’s message.
• Use overlays, built-in templates, and segmentable content blocks to control precisely who sees what, without relying on your development team.
• Build enriched visitor profiles in the background that feed directly into smarter segmentation across your email campaigns and other channels.
• Connect seamlessly with our CDP, email marketing automation, and the broader Spotler Marketing Cloud for B2B, so your testing insights improve every touchpoint, not just the landing page.

If you want to move beyond one-size-fits-all landing pages and start converting more of the right visitors, speak to our team about what Spotler Website Personalisation can do for your campaigns.