You’ve changed a product page, sales went up the next week, and you’re not sure whether the change did it or whether it was payday. That’s the problem A/B testing solves. It shows two versions of something to different visitors at the same time, so the only real difference between the groups is the thing you changed.

You don’t need a statistician to do this well. You need a clear question, enough patience and a few rules. This guide gives you all three.

What an A/B test actually is

In an A/B test, you split visitors randomly into two groups:

• Group A sees the current version (the “control”)
• Group B sees the new version (the “variant”)

Both run during the same period, so seasonality, ad campaigns and paydays affect both groups equally. At the end, you compare a single metric, such as the percentage of visitors who placed an order.

That’s different from a before-and-after comparison, where you change something and compare this month to last month. Before-and-after is easy to fool yourself with, especially in stores with seasonal sales.

Is your store ready to test?

A/B testing needs traffic. With too few visitors, random noise looks like a result. There’s no universal threshold, but here’s a rough way to think about it:

• If a page gets only a handful of orders a week, a test on that page could take months to show anything reliable.
• If a page gets hundreds of orders a week, you can test small changes and get answers in a few weeks.

If your traffic is low, don’t give up on testing. Instead:

1. Test bigger changes. A completely different product page layout is more likely to produce a visible difference than a new button color.
2. Test on your busiest pages, like the homepage, the top product page or the cart.
3. Measure a metric closer to the change, such as add-to-cart rate instead of completed orders, since there are more add-to-carts than orders.

Step 1: Start with a problem, not an idea

Weak tests start with “Let’s try a green button.” Strong tests start with something you’ve observed:

• “Lots of visitors reach the product page but few add to cart.”
• “Customers keep emailing to ask about delivery times.”
• “Many people add to cart but leave at the shipping step.”

Look at your analytics funnel, read your support emails and watch a few session recordings if you have them. Our guide to the ecommerce conversion funnel explains how to find where people drop off.

Step 2: Write a hypothesis

Turn the problem into one sentence:

Because [observation], we believe that [change] will cause [effect], measured by [metric].

For example:

Because many customers email asking about delivery times, we believe that showing the estimated delivery date next to the Add to cart button will increase add-to-cart rate on product pages.

This keeps you honest. You decide the metric before seeing results, so you can’t pick whichever number happens to look good afterward.

Step 3: Pick one primary metric

Choose one number that decides the winner. Common choices for WooCommerce stores:

Revenue per visitor is often the best overall metric, because it catches cases where a variant gets more orders but smaller ones. It needs more traffic to be reliable, though.

Also keep an eye on one or two “guardrail” metrics you don’t want to damage, like average order value or refund rate.

Step 4: Choose your tool

Google Optimize was shut down in September 2023, so many small stores lost their free testing tool. Today, your options fall roughly into three groups:

• WooCommerce-specific plugins that test things like product pages, offers and popups using your store’s own order data. iConvert Optimizer includes A/B testing designed around WooCommerce, alongside its recommendation and cart recovery features.
• General WordPress A/B testing plugins that test pages, blocks or headlines.
• Standalone testing platforms that work on any website via a script. These are powerful but can be expensive and add weight to your pages.

Whatever you use, check that it:

• Splits visitors randomly and consistently (a returning visitor sees the same version)
• Tracks actual WooCommerce orders and revenue, not just clicks
• Doesn’t cause a visible “flicker” where the original version loads first
• Works with your caching setup

Step 5: Decide the duration before you start

This is where most tests go wrong. Rules of thumb:

• Run for full weeks. Weekday and weekend shoppers behave differently. Run for at least one week, ideally two or more.
• Don’t stop early because one version is ahead. Early results bounce around. A variant can lead for days and then fall back.
• Don’t run through major sales events unless you’re testing something for that event. A Black Friday week will distort a normal test.

Many testing tools include a sample size calculator. Use it before launching, and write down the planned end date.

Step 6: Read the results properly

When the test ends, your tool will usually report a winner along with a confidence or significance figure. Treat it as a guide:

• Clear winner with good confidence: Roll it out.
• No clear difference: That’s a result too. The change doesn’t matter much, so choose whichever version is easier to maintain.
• Variant is worse: Good to know before you rolled it out to everyone.

Then check your guardrails. A variant that raises conversion rate but drops average order value may not be a win.

A worked example

Here’s a hypothetical test for a store selling outdoor gear.

Observation: The support inbox gets many “is this waterproof?” questions about jackets.

Hypothesis: Adding a short “Key specs” list (waterproof rating, weight, packable yes/no) above the Add to cart button will increase add-to-cart rate on jacket pages.

Setup:

• Pages: the 12 jacket product pages
• Traffic: say around 4,000 visitors a week across those pages
• Primary metric: add-to-cart rate
• Guardrail: average order value
• Duration: planned for three full weeks

Result after three weeks (hypothetical numbers):

The testing tool reports high confidence that B is better, and average order value is unchanged. The store rolls out the specs list to all jackets, then adds it to other categories and tests again there, since what works on jackets won’t necessarily work on accessories.

What to test first

If you’re not sure where to begin, these areas tend to produce visible differences because they address real doubts shoppers have:

1. Delivery and returns information near the Add to cart button
2. Product page image order (lifestyle photo first vs product-on-white first)
3. Product description structure (short benefits list vs long paragraph)
4. Free shipping threshold messaging in the cart
5. Popup offer type (discount vs free shipping vs a useful guide)
6. Cart page recommendations (none vs a small set of relevant add-ons)
7. Checkout field count (removing optional fields you don’t need)

Our product page design anatomy is a good source of ideas for product page tests.

Mistakes that waste your traffic

• Testing several changes at once in one variant, then not knowing which one mattered. It’s fine if you just want a better page, but you learn less.
• Running too many tests on the same visitors at the same time, so the tests affect each other.
• Ending the test on a good day.
• Ignoring mobile. Check results by device. A layout that wins on desktop can lose on phones.
• Not writing anything down. Keep a simple spreadsheet of each test: hypothesis, dates, result and what you did next. After a year, it’s one of the most useful documents your store has.

Start with one test this month

A/B testing is less about tools and more about habits: start from a real problem, decide the metric in advance, run for full weeks and accept what the numbers say. One well-run test a month adds up to twelve solid lessons about your customers by the end of the year.

If you’d like testing that’s already connected to your WooCommerce orders, iConvert Optimizer is one option worth a look.