Measuring whether the journey improved after you changed it

4 days ago
3 min read

Updated: 2 days ago

Introduction

You mapped the journey, found a problem, changed something. Three months later somebody asks whether it helped, and the honest answer is that nobody knows, because no measurement was agreed before the change and the numbers that exist now have nothing to be compared against.

This is avoidable and cheap to avoid. It requires one decision at the start — which number should move, over what period — and a note of where that number stood on the day. Everything difficult about evaluation comes from skipping those two minutes.

1. Measuring whether the journey improved starts before the change

The sequencing point.

Write down the current value of the number you expect to move, and the date. Retrospective baselines are guesses and they always flatter the change. This is the single most valuable habit in the whole exercise. Two minutes, once.

2. Choose one number, not a dashboard

The focus point.

If four numbers move in different directions you will argue about interpretation rather than reach a conclusion. Pick the one the fix was aimed at. Watch the others, but decide on one. A single measure also makes the review short enough to actually hold.

3. Prefer the number closest to the change

The proximity rule.

If you shortened response time, measure response time first and conversion second. The nearer measure moves sooner and more visibly. The distant one confirms, later, that the near one mattered. Waiting only for the distant one tells you nothing for months.

4. Decide the period in advance

The patience point.

Some changes show within a fortnight and some take a full sales cycle. Agreeing the window beforehand prevents both premature declarations of success and indefinite waiting. Write the review date down with the baseline. A change is fairly judged over the length of your own buying cycle, not over a calendar quarter.

5. Elapsed time is often the clearest signal

The practical measure.

At small volumes, conversion rates bounce around too much to read, while the time between stages is stable and countable. It also tends to be what you actually changed. Start there when the numbers are small.

6. Account for what else changed

The honesty requirement.

Seasonality, a price change, a new channel and a staff departure all move the same numbers. Note what else happened in the period. A result you cannot separate from a busy quarter is not yet a result.

7. Ask the people who see it daily

The qualitative check.

Whoever handles enquiries knows whether the process improved before any number confirms it. Their account is evidence, not colour. Where the number and the account disagree, look harder at both.

8. Accept a null result

The discipline.

Some fixes do nothing, and finding that out cheaply is a good outcome rather than a failure. It removes a candidate and points at the next one. Businesses that cannot record a null result stop measuring entirely.

9. Record the answer where the map lives

The continuity point.

The next revision should be able to see what was tried and what happened. Two lines beside the stage is enough. Without them the same fix gets proposed again in a year.

Be careful about judging the mapping exercise by whether the first fix worked. The exercise produced a ranked list of candidates; the first one failing simply means the second is next, and that list is the durable output.

Conclusion

Write down the number and its current value before you change anything.

Choose a single measure rather than a dashboard, prefer the one closest to what you changed, agree the review period in advance, use elapsed time when volumes are too small for conversion rates to be readable, note what else changed during the period, ask the people handling enquiries what they saw, record a null result as readily as a positive one, and keep the answer with the map so the next revision inherits it.