Counting the bricks, missing the mortar

There is a smell in the air of engineering management in December. Cold coffee. A template for reviewing people. Maybe...maybe there’s a tool that will just tell us who’s been good this year?

There are tools. There are tools that count pull requests, commits, review comments by individual. There are tools that make charts. Charts are answers. This year seems especially susceptible to the tool pull. Budgets are constrained. Layoffs recently happened. People up the chain want to know who’s good. We could use a number.

I keep going back to this brick wall. Say you want to know who made a wall. Count the bricks individual people put in. It’ll tell you something. But not who mixed the mortar, or who noticed that the foundation was off before anyone started laying. Not who took a morning to help the new person get the rows straight. The wall is all of those things. The brick count only measures one thing.

So consider a typical week for a good senior engineer. They make a review comment that stops a bad schema from going live. They spend an hour next to a person with a shaky test. They wrote up a design doc that saves a couple people a week of work. Maybe they do a pull request, if the week is lucky. They’re perfectly normal on the commit chart. They look like they’re sleeping in a bit. They might be why things went okay this quarter, for what the team really cares about.

Now consider the person at the top of the commit chart. Maybe they’re great. Maybe they’re breaking the work down into tiny little parts and avoiding any thorny parts while silently generating lots of review work for others. Chart can’t distinguish the two. Not a great feature of a chart.

What I really hate is what happens afterwards.

Once everyone realizes the number is there to rank them they start bending to the number. Less pairing. Skimmier reviews. No offers to do the hideous migration that will take a month and add two commits. A year goes by and the team has gradually reformed to the measure. They’re worse at the important things. None of that is a conscious decision. It just happens, like how water settles in low places.

I don’t hate data. In fact, I love it.

But more for me it’s pointing it at the team. Are we shipping the things we said we’d ship. Are they useful? How long until changes make it to people? How often are we breaking stuff? How fast are we fixing stuff?

And for individuals I find myself using something a bit less structured. A series of specific moments over the year. And what the peers say about who helped them out of trouble.

I think activity counts can be useful. Almost like a smoke alarm rather than a scoreboard. A month without merging anything, and you’ve maybe got to ask them about it gently. Could be they’re stuck on some project. Could be there’s family stuff going on. Could be nothing.

This doesn’t lend itself to dashboards, and it uses more of a manager’s time than I’ll admit. Maybe that’s ok. Maybe it’s not fitting that’s the point.

I do sometimes wonder if the high brick counters know who’s stirring their cement. I’d bet most of them do.