Most A/B testing advice is written for companies with a growth team, a data analyst and a few hundred thousand visitors a month. If you run a five-page website for your landscaping business, very little of it applies to you. Some of it is actively unhelpful.
This guide is for everyone else. It covers what A/B testing actually is, what you can realistically get out of it with small-business traffic, how to read a result without a statistics degree, and when you're better off not testing at all. It's long because it's meant to be the page you come back to. Each section links to a more detailed post if you want to go deeper.
What A/B testing actually is
Say you run a bakery that takes custom cake orders through its website. Your homepage headline says "Welcome to our bakery." You suspect something like "Custom birthday cakes, ordered online in two minutes" would get more people to fill in the order form.
You could just change it and see what happens. The trouble is that "what happens" gets tangled up with everything else going on. Orders might go up because school holidays started, or down because it rained all week. You'd never know whether the headline did anything.
An A/B test gets around that by running both versions at the same time. Half your visitors see the old headline (version A) and half see the new one (version B). Both groups arrive in the same weeks, under the same conditions. After a few weeks you compare how many people in each group filled in the order form. If version B's group ordered noticeably more, the headline is the likely reason, because it's the only thing that was different.
That's the whole idea. A testing tool does the mechanical parts: splitting visitors between versions, making sure a returning visitor sees the same version again, counting results, and telling you when the difference is big enough to trust. You decide what to test and what to do with the answer.
You only need a few terms. A "version" (some tools say "variant") is one of the options you're comparing. The "goal" is the action you're counting, like a form submission, a button click or a booking. The "conversion rate" is the share of visitors who complete that goal. That's most of the vocabulary.
Why it's worth doing on a small budget
Small businesses usually try to grow by getting more visitors: ads, SEO, posting on Instagram, chasing Google reviews. All useful, and all of them cost time or money every month. Testing works on the other side of the ledger. It gets more out of the visitors you already have.
The maths is simple. If 1,500 people visit your homepage each month and 2% of them send an enquiry, that's 30 enquiries. Lift the rate to 3% and you get 45, without spending another dollar on ads. If an average job is worth $400 to you, those extra 15 enquiries are worth chasing even if only a third of them turn into work.
The second reason is less obvious. Most small business websites are built on opinion: yours, your web designer's, and your brother-in-law's, who thinks the photos are too dark. Testing replaces the argument with an answer. Even when a test comes back flat, you've learned that the change didn't matter, so you can stop worrying about it and move on.
What you can realistically test with small-business traffic
This is the section most guides skip, and it's the one that matters most for a small site.
A test needs enough visitors to tell a real difference from random noise. The less traffic you have, the bigger the difference has to be before a test can spot it. Here are some rough numbers for a page that converts at 3%, with a test that runs for four weeks:
- With 1,500 visitors a month to the page, a test can reliably pick up a change that lifts conversions by around 85% (from 3% to about 5.5%).
- With 5,000 visitors a month, that drops to around 45%.
- To reliably confirm a 10% improvement, you'd need roughly 84,000 visitors. At 1,500 a month, that's about four and a half years.
Those numbers can look discouraging. Read them another way: on a small site, A/B testing finds big wins, not small ones. Changing a button from "Learn more" to "Read more" will never produce a result you can see. Rewriting a vague headline, cutting a ten-field form down to four fields, putting your prices on the page or moving your phone number to where people can find it can all produce big differences. On most small business websites, those changes haven't been made yet.
Two other things help. First, test on the page with the most traffic, which is usually your homepage or main landing page. The number that counts is visitors to that page, not your whole site. Second, if your main goal happens rarely, measure something that happens more often. Clicks on a "Get a quote" button happen far more often than finished quote requests. A goal that converts at 15% lets the same 1,500 visitors detect a lift of about a third, instead of 85%.
The full breakdown by traffic level is in How much traffic do you need for A/B testing? If the page you'd test gets fewer than about 500 visitors a month, read that first.
What to test first
If you only run one test this year, make it your homepage headline. Every visitor sees it, and on most small sites it's vague enough that a specific rewrite has real room to win.
"Welcome to our website" and "Quality service since 1998" tell a visitor almost nothing. "Emergency plumbing in Parramatta, at your door within the hour" tells them straight away whether they're in the right place. The gap between those two is exactly the kind of big difference a small site can detect.
After the headline, the usual candidates, in rough order, are the main call to action (both its wording and where it sits), the length of your contact or booking form, what's in the hero area at the top of the page, and where your reviews sit relative to the button. Test one thing at a time, and make each change bold enough to matter.
There's a longer list, ranked by effort and likely impact, in What to A/B test on your website.
If you run a service business
Most A/B testing content assumes you sell products online, so it talks about checkout flows and add-to-cart buttons. If your website's job is to get someone to call, book or fill in a form, that advice needs translating.
The main difference is what you count. For a service business the goal is usually a form submission, a booking, or a tap on your phone number. Phone calls are the awkward one, because a visitor can read your number off the screen and dial it from another phone. Counting taps on a click-to-call link gets you most of the way there, especially on mobile.
Visitors behave differently too. Someone with a burst pipe wants a phone number and some proof you'll turn up today. Someone choosing a family lawyer reads three websites over a week before calling anyone. Your tests should match the visitor you actually get: speed and urgency for the first, reassurance and credentials for the second.
A/B testing for service businesses goes through specific tests for trades, health practitioners, lawyers and other service businesses, with examples.
How to read a result without a statistics degree
At the end of a test you'll see something like this: version B converted at 4.8%, version A at 3.1%, with 93% confidence. The first two numbers are easy. The third is the one people misread.
Confidence is roughly how sure you can be that B really is better and the gap isn't luck. SplitPea calls a winner at 90%. Most enterprise tools default to 95%, which is a sensible standard when a wrong call costs a lot of money. When you're choosing between two headlines, being right nine times out of ten is plenty, and the lower bar means you need less traffic to get an answer.
A few habits keep your results honest.
Decide how long the test will run before you start. Two weeks is the minimum, so you capture full weekly cycles. Allow up to six weeks if your traffic is modest.
Don't call it early. In the first few days, one version will often look like it's running away with it. That's small numbers bouncing around. A version that's "winning by 40%" on day three can be dead level by day fourteen.
Treat "no clear winner" as an answer. If a test runs for six weeks without reaching confidence, the two versions perform about the same for your visitors. Keep whichever one you prefer and test something bolder next.
Write down what you decided and why. In six months you won't remember whether the shorter form actually won or just looked like it did.
For a plain-English explanation of the confidence number, read What 90% confidence actually means.
One test, start to finish
Back to the bakery. Here's what a realistic first test looks like, with realistic numbers.
The homepage gets about 2,000 visitors a month. Only around 1.5% of them finish the cake order form, which would mean roughly 30 orders a month and far too few to test against. But about 12% of visitors click the "Order a cake" button, so that click becomes the goal. It happens often enough to measure, and it's the step right before the order.
The hypothesis is written down before anything changes: "A headline that says what we make and how easy ordering is will get more people to click through to the order form." Version A keeps "Welcome to our bakery." Version B says "Custom birthday cakes, ordered online in two minutes." Nothing else on the page changes. The test is set to run for four weeks.
In the first week, version B is ahead by 60%, and it's very tempting to call it there. By the end of week two the gap has shrunk to about 20%, which is why you don't call tests early.
After four weeks, each version has had about 1,000 visitors. Version A got 118 clicks (11.8%) and version B got 146 (14.6%). That's a 24% lift with 97% confidence, comfortably past the 90% bar. Finished orders point the same way, though they're too few to prove anything by themselves.
So version B becomes the headline. The decision goes in the log with a line about why it probably worked: it tells visitors exactly what the bakery makes and removes the worry that ordering will be a hassle. The next test is the order form, which currently asks for nine pieces of information before anyone sees a price.
None of this needed a statistician or a developer. The steps were to pick the page with the traffic, pick a goal that happens often enough, make one bold change, wait, and write down what happened.
When A/B testing isn't worth it
We sell an A/B testing tool, so it's worth saying plainly that testing isn't always the right move.
If the page gets fewer than about 500 visitors a month, a test will take months to say anything useful, and your business will have moved on by the time it does. If something is obviously broken, like a form that errors on mobile or a page that takes six seconds to load, just fix it. You don't need a test to know broken is worse. And if you're about to redesign the site, wait until the new one is live. Testing a headline on a page you're replacing next month is wasted effort.
For very small sites there's a middle ground between testing properly and guessing. Make one change, write down the date, and watch the goal you care about for a few weeks before and after. That's weaker evidence than a test, because anything else that happened that month (a public holiday, a new review, an ad you paused) gets mixed in. It's still far better than changing things and never checking. Once your traffic grows, your notes will tell you which questions deserve a proper test.
Some things are worth doing without any testing at all. Ask five people who don't know your business to look at your homepage and tell you what you do and what they'd click next. Start from the defaults that are almost always right: a specific headline, a visible call to action, a short form and reviews near the button. And put real effort into getting more visitors, because that solves the testing problem along with a lot of others.
Where SplitPea fits
We built SplitPea because when Google Optimize shut down, the cheapest replacements started at around $200 a month. That's a lot to spend on finding out whether your homepage works.
It's made for the kind of site this guide describes. You add one small snippet to your site, click the element you want to change in the visual editor, type the new version, pick the goal you're counting and start the test. SplitPea splits your traffic, keeps returning visitors on the same version, and tells you in plain English when there's a winner (and when there isn't). Every result goes into a decision log, so you keep a record of what you tried. Between tests, the same snippet keeps tracking your goals over time, which makes the before-and-after checks above much easier.
The Starter plan is free for one website and 500 visitors a month, which is enough to set up a test and see how it all works. Grow is $29 a month for 5,000 visitors across three sites, and Agency is $49 a month if you manage client websites. The pricing page has the details. If you're still comparing tools, our guide to Google Optimize alternatives covers the other affordable options fairly, including the ones that might suit you better. If your site runs on Squarespace or is a static site, there are setup guides for both.
Where to go from here
You now know enough to run a useful first test. Pick your highest-traffic page, write a bolder headline and run both versions for at least two weeks.
Your first A/B test: a five-minute setup guide walks through it step by step.