How to test changes when you do not have thousands of visitors

Most testing advice was written for sites with heavy traffic. Applied to a business receiving a handful of enquiries a week, it produces confident conclusions from numbers far too small to support them.

Talk to us about your site

At small volumes, formal A/B testing usually cannot reach a reliable answer in a useful time. Instead: change one meaningful thing at a time, record the date, compare like periods including the same season last year, and prefer changes that are obviously right on their own terms. Treat small movements as noise rather than evidence.

Why split testing struggles at small volumes

A split test works by dividing visitors between two versions and waiting until the difference between them is larger than random variation. The fewer conversions you have, the longer that takes, and for many small businesses the honest answer is months per test.

By then the season has changed, a competitor has moved, and you have run one test instead of making ten obvious improvements. The method is not wrong, it is simply built for a scale most small businesses do not have.

The habit that works instead

Sequential and disciplined beats simultaneous and statistical here. Change one meaningful thing, write down what and when, and let it run long enough to see a pattern.

  • One meaningful change at a time, not six in an afternoon.
  • A dated note of every change, kept in one place.
  • Enough weeks to see a pattern rather than a good fortnight.
  • Comparison against the same period last year where seasons matter.
  • A written expectation before the change, so you cannot rationalise afterwards.

Prefer changes that do not need proving

A great many improvements are obviously right and need no test at all. Making a phone number tappable. Explaining what you do at the top of the page. Compressing an image that weighs several megabytes. Replying to enquiries the same day.

Spend your limited measuring capacity on genuinely uncertain decisions, and simply make the obvious improvements. Testing whether a broken thing should be fixed is a waste of months.

Beware conclusions drawn from tiny numbers

Going from three enquiries to five is not a large increase in evidence, however impressive it looks as a percentage. It is well within the range of ordinary variation for most small businesses, and building a strategy on it is how businesses end up chasing their own noise.

A useful discipline: never express a change as a percentage when the underlying counts are small. Say five enquiries against three last month, which is honest and makes the fragility obvious.

Watch for the things that were not your change

Enquiries move for many reasons that have nothing to do with your website: a season, a competitor's advertising, a search engine update, a local event, a holiday period, or simply luck.

Keeping the dated note of changes helps here, because it lets you check whether a movement lines up with something you did or with something that happened to everybody. Where you can, look at whether the whole market moved rather than only your site.

Qualitative evidence is real evidence

With small numbers, watching five people use the site tells you more than a statistical test you cannot finish. Where they hesitate, what they misread, what question they ask you at the end: these are findings, and they are usually unambiguous.

Enquiries themselves are also data. If three people this month asked something the website should have answered, that is a page to write, and no test is required to know it.

How Licheo works with this

We change things in a deliberate order, record what changed and when, and report against outcomes rather than percentages calculated from small counts.

Where a figure is not reliable enough to draw a conclusion from, we say so. An honest gap is easy to fill next quarter; a confident conclusion drawn from four enquiries can misdirect a business for a year.

Part of a larger guide

This page is one part of More revenue, same traffic. The other parts:

Questions people ask

Can I A/B test with low traffic?
Rarely in a useful timeframe. With few conversions, a split test can take months to separate a real difference from ordinary variation. Sequential changes with dated notes usually teach you more within the same period.
How long should I leave a change before judging it?
Long enough to see a pattern rather than a good week, which for most small businesses means at least a month or two, and longer where the sales cycle is long or the season is strong.
How do I know a change caused the result?
You often cannot be certain, so reduce the alternatives: change one thing, note the date, compare like periods, and check whether anything else moved at the same time, such as a season or a competitor's campaign.
Should I use percentages when reporting small numbers?
Better not to. A move from three enquiries to five reads as a large percentage and is well within ordinary variation. State the counts, which is honest and makes the fragility of the conclusion visible.
What counts as evidence besides numbers?
Watching a few people use the site, and the questions your enquiries keep asking. If three people this month asked something your website should have answered, that is a page to write, and no test is needed.

Improve deliberately, measure honestly

One change at a time, dated, reported against real outcomes, with no percentages invented from small numbers.

Talk to us about your site Read the full guide