udae
Technical notes/MethodologyMetricsROI

How we measure before and after: the only proof a project delivered

No baseline number, no project. And without a follow-up number, no proof either. The improvement of an operational system is demonstrated with numbers, not claimed. How we define the metrics before touching anything.

Álvaro Urra··7 min
Instrumento de medición

The first number in a project is not the budget. It's the number that describes the problem. Without it, there's no project: there's a style exercise and the hope that things will get better. At udae we don't start an automation without taking that number first. And we don't close it without taking it again at the end. This note explains how we do it.

The two-photo rule

The idea is simple: two photographs with the same camera, framing, and settings. One taken before touching anything. The other, taken a month after delivering the system. Everything that happens between the two photos is methodology negotiation, tool selection, and technical decisions. But the project is judged by the difference between them.

This approach isn't original. Any industrial engineer will recognise the baseline and post-implementation pattern. What is ours is the requirement: without those two photos, there's no project. No "you'll notice it". No "I promise". A number before, a number after. If it doesn't improve, it didn't work.

What we measure

It depends on the process, but almost always some combination of three things:

  • Human time. How many minutes of work a person spends on the task, how many times per day or month. Multiplied by the hourly cost of the role. In reconciliations, payments, notifications, and reports, this is the dominant metric.
  • Catchable errors. How many times per month a discrepancy is discovered late: an unpaid invoice, a duplicated payment, a customer complaining about a service not recorded. Every error has a direct cost, and an indirect one in trust.
  • Decision lag. How many days pass between a fact happening in operations and it appearing on the report leadership sees. In many businesses it's the quietest and most expensive metric.

We don't measure more than three or four things per project. More metrics mean less focus, less rigour, and above all more excuses for not delivering.

How the "before" number is taken

This is where projects usually get stuck. The company wants to start; the technical team wants to build; leadership wants to see progress. Asking for two weeks to measure the current state feels like a luxury. It isn't. It's the only way to know whether the system to be built works.

Two weeks is the reasonable minimum. One month is ideal. In that period we ask the team to record, without changing anything, three data points per process instance: when it started, how long it took, and whether there was an error. We don't ask for a meticulous diary: a shared sheet where each person notes theirs is enough. The final number is an average with an uncertainty band, not an exact value.

Real example

Before: monthly financial close for three clubs. 3h 40min per close. 4 errors caught per month. Report to leadership with a 5-day lag.

This was the starting point of a project we did with a three-gym group. Measurement took three weeks. With those numbers on the table, the proposal changed completely: we didn't propose migrating from Virtuagym, nor changing the finance software. We proposed a flow that read Virtuagym and Mollie, reconciled automatically, and published the report on a panel leadership pulls from whenever they want.

Four months after delivery, the second photo:

After: monthly financial close for three clubs. 12 min per close. 0 errors caught in three months. Report to leadership available every morning.

The conversation changed. Before we talked about the project in the abstract. After we talked about what to do with the three weekly hours the office had recovered. That's the only kind of conversation that justifies a project like this.

When the numbers don't improve

It happens. It's not common, but it happens. And at udae we treat it with the same discipline as when they do improve: look at the photo, compare the numbers, and admit it. There are three common reasons a project doesn't move the needle:

  1. The wrong metric was measured. The system improved something real, but not what was promised. It's a project design error, not a system one. Fixed by picking the right metric from the start.
  2. The automated process wasn't the one consuming the time. The team spent the hours on another task that wasn't in scope. Detected in the first measurement if done properly; if it appears later, scope must be extended.
  3. The system isn't being used. Nobody pulls from the panel, nobody looks at the report, nobody trusts the flow. It's an adoption problem, not a software one. Fixed with training, not more code.

The second photo is what commits

The first photo is taken by the project. The second is taken by the client, with our support. It's them who access the panel, look at the records, and count the minutes. If the number doesn't improve, the conversation isn't pleasant. But it's the only conversation that matters. Everything else is literature.

That's why, when someone asks for a project without wanting to measure, the answer is usually not yet. First we need to define how we'll know it worked. Then, and only then, do we build.