I lead product at Routespring, a business travel management company built for airlines to manage their flight crew travel.
Our customers are carriers, and the travelers are their crews: the pilots, first officers and flight attendants who have to be in the right city, rested and legal to fly, before an aircraft can go anywhere. Most of what we build is the unglamorous machinery underneath that. Booking, policy, payment, reporting, the parts nobody notices until they break.
It looks like corporate travel software from a distance and it behaves nothing like it. Ordinary business travel is planned weeks ahead by the person taking the trip. A lot of crew travel is not planned at all. It is what happens after something has already gone wrong.
This is the story of one feature we are building there, and specifically of how I decided what "done" was going to mean for it. That second part took me longer to get right than the technology did.
The problem
A flight cancels at nine in the evening, or diverts to an airport nobody planned to land at.
The passengers are a problem, and they are not this problem. The crew is. Pilots, first officers and flight attendants are on a clock. They are legally required to rest before they can fly again, and that rest has to happen in a real hotel bed. So a room here is not a courtesy and it is not an expense line. It is the thing that decides whether the first flight out in the morning leaves at all. One crew without rooms is an aircraft that does not move, and a schedule that starts unravelling in three cities by lunchtime.
Somebody has to put that specific crew into rooms near that airport, tonight, at an hour when most of the city is asleep.
And it turns out to be a phone job rather than a search box. What matters at 11pm is not what a hotel's website lists. It is what the front desk will actually hold right now, at what rate, for that many people, tonight. The person who knows that is the person picking up at the desk. So a coordinator works the phone, hotel by hotel, one call at a time, at exactly the hour when they are already handling everything else the disruption just created.
That is the job. It is serial, it is manual, and it happens at the worst possible moment.
What we expected the agent to do
The brief was narrow on purpose.
The agent calls a set of hotels at once. It asks what is available tonight and at what rate. It reports back to the coordinator, who decides. It does not book anything, it does not negotiate anything it was not told to, and it does not talk to the crew. It removes the part of the job that is dialing, sitting on hold, and starting over, and it leaves the judgment exactly where it already was.
On paper this is one of the easier conversations you could hand a voice model. It is short, it is scripted, and it has one question in it.
Then you listen to the calls.
A hotel line answers with an automated menu, or a voicemail greeting, or a person who picks up halfway through their own sentence and says "front desk" so fast it lands as a single word. Sometimes hold music that sounds like somebody about to speak. Sometimes a menu that hands you to a human with no warning at all. The agent has to work out which of those just happened before it opens its mouth, and it has about a second, because the alternative is talking over a live person who is already having a bad night.
Early on, roughly one call in ten came back with the answer we were calling for.
My first definition of done was useless
My instinct was the obvious one. Measure the outcome. Did the call succeed, yes or no.
That gave me a clean number. It went into status updates. And it was useless.
Useless because all nine failures looked identical from the outside. A call that failed because the agent talked over a menu prompt and a call that failed because it reached exactly the right person and then fumbled the question are the same row in that table. The number was accurate, it was honest, and it had nothing to say about what anybody should do on Monday.
This is the failure mode I have come to worry about most, and it is not the one people warn you about. A vanity metric flatters you. This one flattered nobody and still pointed nowhere.
So I stopped defining done, and started sequencing it
The change was to stop asking whether the call succeeded and start asking what had to be true first.
Gate one: tell a live human apart from an automated menu. Cleanly. One hundred percent of the time. Nothing else counted until that held.
Not mostly. Not in the common cases. One hundred percent sounds unreasonable until you notice what it buys, which is every number that comes after it. If the agent cannot reliably tell who is on the other end, then every measurement downstream is polluted by a mistake made in the first second of the call.
Then: get through the menu. Then: a clean introduction to the person who picks up. Then the thing I actually wanted, which is the call coming back with an answer about availability.
Each gate is narrow enough to be right or wrong. That property is the whole trick. Not the ambition of the target, the narrowness of it.
The rule that makes it work
While gate one is still failing a third of the time, gate two does not have a number. It has noise.
Measure the menu handling on top of an unsolved answer-detection problem and you cannot tell whether the handling is bad or whether it simply never got a fair run. You will spend a week improving something that was working the whole time. I know because I have done it.
So a stage is not measurable until the one below it is boring. Boring is the word I use for it. A stage is finished when it stops producing surprises, and only then does the next one become a question you can actually ask.
That also answers the thing people ask me first, which is how you know a definition is the right one before you build on it. The test is built in. If a gate is right, clearing it makes the next stage measurable. Clear it and find that nothing downstream got easier to see, and you measured something real that did not matter. You get that answer in one stage instead of one quarter.
Why one gate is 100% and the next is 50%
The end-to-end gate on this agent is not one hundred percent. It is half. The feature does not enter our development pipeline until half of its calls come back with an answer about availability.
That number needs one clarification, and insisting on it was probably the most useful thing I did.
It does not measure whether the call went well. It measures whether we ended up knowing what rooms were available. Those are different, and the gap between them is where the interesting failures live. Picture a call where the agent detects the menu, works through it, reaches the front desk, introduces itself properly, and is politely asked to call back in an hour. Nothing went wrong. As a conversation it is a complete success. It is also a zero, because we still do not know whether there are rooms, and the coordinator still has to make that call themselves.
Had I graded call quality instead, that call would have counted, the number would have looked better, and the feature would have been worth exactly as much as it was before. The easy thing to check was how the agent behaved. The thing that mattered was whether anybody learned anything.
That is also why fifty and not one hundred. Some of the ways a call fails to produce an answer have nothing to do with the agent and never will. The desk is alone with a queue of guests. The person who can check availability is on a break. A hotel simply will not quote to a machine. No engineering budget drives those to zero, so a bar of one hundred percent there would not be ambitious, it would be a category error.
Nothing about fifty is scientific and I would not defend the exact figure. What it does is give everyone the same picture of a problem that is almost impossible to see from a demo. A recording of one good call tells you the thing works. A rate tells you what a coordinator would actually get on a bad night. We stopped estimating readiness and started counting it, which turns out to be a much easier thing to be patient about.
It also means the honest status of this feature today is that it is not shipped and not in the pipeline. That is not a delay. That is the gate doing exactly what it was built to do, which is to stop us from spending engineering capacity on something that is not ready to absorb it.
Then retire the gate
Once a gate has held for a quarter, it has done its job.
Most teams do the opposite. The check stays on the dashboard, permanently green, quietly congratulating everybody. What that green square actually represents is a question that got settled a while ago, and a slot you are no longer using. Meanwhile the live risk, the thing that could genuinely sink the feature this month, has no number on it at all, because nobody moved the gate.
A definition of done that never changes is not a standard. It is a souvenir.
What it cost
Two honest things.
I defended my own first definition longer than it deserved. Writing a check is a decision, and decisions attach themselves to the person who made them. It took me an uncomfortable amount of time to accept that a target I had set was the thing holding us up.
And you cannot import somebody else's ladder. Ours encodes a pile of assumptions about phone systems, about what a front desk sounds like at 11pm, about which failures are recoverable mid-call. None of that would be the first thing I would write down for a different product. The visible artifact travels easily. The fit does not, and the fit is most of the value.
Where to start
Take the thing your team currently calls hit or miss.
Do not try to define success for it. Define the first thing that has to be true before success is even a sensible question, and make it narrow enough that it can only be right or wrong. Then watch whether clearing it makes the next question askable.
The definition of done was never a line you cross. It is a ladder, and the discipline is not writing the first rung. It is throwing that rung away once you have stopped falling off it.
I made the wider argument for why this is a strategy question rather than a process question here: Your Definition of Done Is Your Product Strategy.
Originally published on X. View the original →