Pricing experiments AI can run for you, safely

5 days ago
3 min read

Updated: 2 days ago

Introduction

Most pricing decisions in small businesses are made once and defended for years. The price was set when the business started, adjusted for inflation occasionally, and nobody has any evidence about what would happen at ten per cent more or ten per cent less. Testing is the obvious answer and it is attempted far less often than it should be.

The reason is not laziness. Pricing experiments have two hard constraints that do not apply to marketing experiments: you cannot charge two comparable customers different prices without a defensible reason, and you need enough transactions for the result to mean anything. Both constraints eliminate a great deal of what is technically possible, and understanding them is what makes the remainder safe to run.

1. Pricing experiments AI can run for you are constrained by fairness first

This is not a technical limit.

Two similar customers receiving materially different prices at the same time, with no published reason, is a problem regardless of what it reveals. Every legitimate design either separates the groups by time, or attaches the difference to a stated condition.

2. Volume determines whether a result means anything

Small numbers produce noise.

A business doing twenty transactions a month cannot detect a five per cent effect in a reasonable period. In that situation, sequential testing across quarters, or testing on a much larger price change, are the only designs that will ever produce a signal.

3. Test by time period rather than by customer

The safest common design.

This quarter at the current price, next quarter at the new one, with seasonality accounted for. It is slower and less precise than splitting customers, and it avoids the fairness problem completely, which is usually the right trade.

4. Attach differences to conditions people can choose

The other clean design.

A discount for annual payment, a premium for expedited delivery, a lower rate for off-peak booking. Everyone can see the condition and anyone can meet it, so the price difference is legitimate and you learn what people will pay for.

5. Start with new customers and new products

Least disruption.

Testing on customers who have no established expectation avoids the worst outcome, which is a long-standing customer noticing a change they did not agree to. New products are the ideal case because nothing has been anchored yet.

6. Measure margin and volume together

The obvious mistake.

A price rise that reduces volume slightly is usually a large net gain, and a price cut that raises volume substantially can still lose money. Only the combined figure answers the question, and it must include the extra cost of serving the extra volume.

7. Watch the second-order effects

They can outweigh the direct result.

Higher prices attract different customers, change your win rate by segment, and alter what your competitors do. Lower prices can raise support cost and shorten retention. The immediate revenue number does not capture any of this.

8. Decide the stopping rule before you start

Otherwise the result is whatever you wanted.

Duration, sample size and the threshold that counts as a difference, written down in advance. Without this, experiments are ended early when the data looks favourable, which reliably produces confident false conclusions.

9. Keep a record of every test and its outcome

The value compounds.

A short log of what was tested, over what period, and what happened, becomes the most useful pricing document in the business within two years. Most firms rerun the same uncertainty repeatedly because nobody wrote the answer down.

Be prepared for the honest result, which is frequently that demand was less price-sensitive than everybody assumed and the price could simply have been higher all along.

Conclusion

Design around fairness and volume, because those two constraints rule out most of what is technically possible.

Separate test groups by time period rather than by customer, or attach the price difference to a condition anyone can choose, start with new customers and new products where nothing is anchored, measure margin and volume together including the cost of extra volume, watch for second-order effects on customer mix and retention, fix the duration and threshold before you begin, and keep a written log of every test and its outcome.