Code Left Unwritten

Of all the bugfixes and features I have delivered recently, there’s one script that particularly stands out. It’s a modest thing, retrieving data from a sequence of scheduled tasks, but it’s more remarkable for what it isn’t.

Every evening a set of tasks aggregate key metrics from across the service ecosystem and cache their output in Amazon S3 for later access. The data lay there largely forgotten, until a question arose that other dashboards and stores couldn’t answer. This dataset wasn’t gathering dust in S3 for a lack of value: rather, it answered a range of salient business questions. But it had fallen victim to a vicious cycle of non-trivial access breeding infrequent access, which–combined with casual AWS users–meant lost keys and paths and even more friction getting to the data.

“Aha!” went the thought. “If we could only have these data available on our team members’ desktops, they could greatly influence our decision-making process.”

I rolled up my sleeves and got to work. In no time at all, I was the proud owner of a Bash script that would teleport the relevant data into the local working directory. It was accessible: given a configured instance of s3cmd, the user would not need to look up any credentials of their own. And it was clever, recursing its way through the bucket to mirror its structure within the local directory.

But as I finished writing out the usage instructions, it struck me: what I had just rewritten already existed. s3cmd sync.

Face, meet palm.

Out went the script. In came its nearly trivial replacement:

s3cmd sync "s3://${BUCKET}"

I couldn’t help grinning. Not for the time wasted on the initial script, of course, but for shipping a solution whose maintenance requirements approach absolute zero.

See, writing software is easy. But pity the developer who writes it without regard to the costs of upkeep and maintenance — and rejoice in those fortunate times they can be avoided entirely!

So stand on the shoulders of giants, with their own documentation, interfaces, communities, and bugfixes. Save your energy for the problems yet unsolved.

Let’s keep in touch

Reach out on Twitter or subscribe for (very) occasional updates.

Hey, I'm RJ: digital entomologist and intermittent micropoet, writing from the beautiful Rose City.