The last twenty percent of a migration

I open up a code base. There are two of them. Two http clients. Two logging libraries. A forgotten orm hanging out in some corner, like a houseplant you don’t water that refuses to die. Two of each. Vestiges of some migration that someone started for good reasons that never got completed.

What I notice isn’t the fact that there is duplicate code, but the fact that every time someone new comes along in their first week they ask, “which one should I use?” And the answer, depending on who you ask, and for how long, can be different.

I’m thinking about the difficulty of the end. Starting a migration is fun. You can write a design doc, debate, write tooling that easily migrate easy callers. You can watch a graph go up in a very satisfying way. People like the graph. People put the graph in slides.

Then the graph flattens. What remains is the long tail: the service with the strange config, the job that runs once a month at three in the morning, the code owned by a team that has since been reorganized into two other teams who each believe it belongs to the other one. Nobody volunteers for this part. There’s no clever tooling left to build, only patient, unglamorous work, one odd case at a time.

It’s worth walking through a made-up one, because the shape is always the same. Say a codebase has four hundred call sites using an old HTTP client, and the plan is to move them to a newer one with better connection pooling and saner timeouts. Somebody writes a codemod, a script that parses the source and rewrites the simple calls. A plain GET with a URL and some headers maps one to one. The codemod runs, a few hundred files change, the tests pass, and by the end of the second week three hundred call sites have moved. The graph looks heroic.

The remaining hundred are where the old client’s personality was hiding. Some of them rely on its retry behavior, which retried on any exception, including ones where the request had already reached the server, so a few POST endpoints have been quietly sending duplicates for years and something downstream has learned to dedupe them. Move those to a client that doesn’t retry and you’ve changed behavior nobody wrote down. Some use a custom interceptor that injects an auth token and refreshes it on a 401. A dozen live in tests that mock the old client’s classes directly, so the tests have to be rewritten before the code can be. A few are in a vendored copy of some internal library that nobody is sure they’re allowed to touch. And one is in a script that runs once a quarter for finance, and nobody knows it exists until it fails.

They each take a day or two. They each require someone who understands the old behavior and who understands the code that invokes it. For the first three hundred, it’s probably a week for a single person. For the last hundred, it’s more than that for each twenty. It’s exactly as cost-per-caller rises as excitement-per-caller falls. Cold blooded, if you think about it.

Which is to say, you sort of get stuck at eighty percent, which is possibly the most expensive number in software. You’re running both. Everyone needs to understand both. Security patches and upgrades for both as language version shifts. Both with their transitive dependencies being pulled in to the build. You can’t fully transition to the new system, which means some of its best features are just theory (can’t use the new client’s timeouts in a shared lib when half the callers are still using the old client).

There’s another kind of subtler but I think underappreciated cost. That the next migration is going to be harder to justify. Teams remember. They’ll politely nod at the next migration. And contribute exactly zero effort. Because the last one didn’t go through. Trust in migrations is a lot like budget. And a partially executed migration is a budget sink. No return.

Maybe you want to keep in mind the reason for doing the migrations in the first place. Mostly nobody goofed. There’s a size of company a tool works well for. And then the company grows out of it. Like a kid outgrows their shoes. In a growing place, you’re going to outgrow things all the time. So it starts to become more important to be good at the whole thing than it is to be good at kickoffs.

‘We’re switching to the new library’ is a feeling. I like to jot down a list of things that will be deleted. Before I start. The old dependency removed from the build. Old config removed. Old docs archived. Seems like the lack of defined end state for a migration leads it to kind of slide into ‘mostly done’ and happily sit in ‘mostly done’ for years.

A lint rule that stops new code from reaching for the old way helps too. The trick I like is a ratchet. Run a check in CI that counts uses of the old thing, keep the current count in a file in the repo, and fail the build if the count goes up. When someone moves a caller, they lower the number. It can only ever go down. It’s a small thing, but at least the tail doesn’t grow back while you’re trimming it, and the number in that file is a more honest progress report than any slide.

The other trick, which I have mixed feelings about, is the shim. You rewrite the old library’s interface as a thin wrapper over the new one, so every remaining caller is technically on the new system and the old dependency can be deleted. Sometimes that’s exactly right. It gets the security and upgrade burden down to one, and it buys time for the weird cases. It also has a way of becoming permanent. Five years later the shim is a third HTTP client with its own quirks, and the new person’s question has a third answer.

I think the real decision is who owns the tail, by name, before the first easy caller moves. The people who built the codemod are the natural choice, since they know both systems best, but they’re also the ones who’ll be pulled onto the next exciting thing the moment the graph looks good. If nobody is on the hook for the last hundred, the last hundred belong to the future, and the future is very busy.

What I really don’t know is how to make finishing feel as good as starting. Organizations throw a little party for a kickoff. Nobody throws a party when the last caller moves and someone quietly deletes a directory. Maybe they should. Cake for the deletion. The old system could get a proper farewell, a card that everyone signs, a few kind words about the years it put in.