Why I Left Backstop and What I'm Doing at Grainger

23 Sep 2022

DALL-E 2: Fear of the Unkown Epic Scene

About a year ago I left my job at Backstop Solutions to accept a DevOps/PaaS position at Grainger. I had been at Backstop Solutions 11 years and, honestly, should have left after 7. By that time it had become abundantly apparent that the company and I had diverged in terms of my project’s importance. What had once been a powerful team of 5 developers, 1 QA, and 1 project manager had dwindled due to the company not hiring replacements for departing team members. At the time of my departure, the team consisted of myself and a junior developer full time on the project. Our QA and management were split between my team and another. The only major project was adapting the application to run in containers instead of virtual machines. In short, it was clear that the project didn’t matter much to my superiors. Speaking of my superiors, the entire management team had changed during my time at Backstop. Some important leadership positions had changed multiple times and not for the best.

I foolishly stayed telling myself I needed to look after the team but, in retrospect, I was comfortable and knew the system inside and out. I liked being the guy who knew things.

Luckily, I still had contacts in the industry and was able to find a place at Grainger. It was and is a huge shift in what I do day to day. At Backstop, I was an application developer who had been thrust into DevOps when the team decided to migrate our hosting to containers. At Grainger, I am part of a team that maintains a numbers of Kubernetes clusters for the application developers. Additionally I write software to enable developers to, for example, setup MongoDB clusters in the cloud. The goal is to simplify the complex networking and authorization parts so that they can get on with business of writing software instead of learning K8s networking/auth/deployment/etc from scratch. I have to admit that the transition has been difficult given the complexity of K8s + Istio and many customizations. Support, in particular, is nuts. Our team ‘owns’ a hundred plus repositories of code and configuration that keep the clusters running. Odds are good that any given incident will take me into a pile code that I have never seen, or even heard of, before. I feel real stupid every time I’m up for support on the rotation.

However, I remember how I felt when I was learning application development. The times I felt dumb were also the times I was learning the most. It’s my opinion that too many people in the software industry find a niche, drill down to bedrock, and build a house with high fences. It is… exactly what I did in my last few years at Backstop. Who doesn’t like being the dude who knows where the bodies are buried? There are vast amounts of developers who have never been on a good team because they’ve only worked at a few lousy code shops. It is common to interview a 10 year developer who is, shockingly, is worse than any number of junior developers. Having to explain to a seasoned coder that “No, declaring a method private is not about security it’s about encapsulation” is something I’ve done far too often. I don’t think it’s a stretch to say that many such programmers only learn increasingly byzantine ways of hiding and protecting themselves from management. Now some jerk shows up and wants openness and one branch and, worst of all, a spy developer attached to their hip called a ‘pair.’ Time to bring out all the obfuscation tricks, right?

A note about the image at the top of this post: I’ve been messing around with AI generated art and this image was created by DALL-E 2 with the prompt “Fear of the Unknown Epic Scene.” I also used the outside generator to add 25-ish percent more to the left side.