“The Breaking of the Fellowship” from The Lord of the Rings: The Fellowship of the Rings by Howard Shore and Edward Ross
Almost 4 years ago when I started this job, Steve Phillips at AWS, who has always been brilliant but finally has been recognized as the Principal (capital P!!! But spelled correctly!) TAM that he is, Steve DMed me with an appropriate meme as I took my new job.
Fair, Steve, fair. We had worked together for almost two years prior to me going to Take-Two when I was at AWS and I ran at the hardest thing I could because (in Ms. Rachel’s voice) “You can do hard things!”
Well, I am hear to say, it’s not kubernetes that’s hard.
It’s the business.
It’s the whole ecosystem.
It’s the teams that do not want to converge on a few ideas but perpetuate many. It’s people being attached to their way of doing deployments – because some want it complicated, some want it simple. And this is everywhere for anyone who owns a cluster in any enterprise company – I wish I could tell you you can run away from it – you can’t. At first, it is exciting. But over time it compounds into maintenance pain and context switching.
It’s the pane (pain?) of glass support channels where “it is always kubernetes’ fault” even though 50% of the time it is not and when it is, it’s actually the CNI. I’m coming to you live from now 4 years of just doing exactly this saying, oh my god, wait, I’m bored? I’m bored of doing Kubernetes. Because eventually, you and your team, you get good at it and you realize that the hard and living in it makes you exceptional (You all are by the way).
Every night I’ve burned all my Claude tokens using Opus in Claude Design just like the rest of you who don’t want to spend your personal budgets on Fable. It’s cool but I have not drunk the koolaid that “everything” and “everyone” can be replaced by AI Agents. Quite the opposite.
Infrastructure is much more nuanced and we should care so much about that craft.
Why wouldn’t we want to agent-fy everything? Oh. Right. Maintenance. Compliance against non-deterministic models.
Look if you ever lived in a Kubernetes cluster you would already know that this story ends at wide diversification and nuance we have to manage instead of centralization, but…I didn’t know if anyone would believe me so I built my 100th website to show us how varied the solutions are for AI Ops against tenancy.
Click Here to Cry In Real time Interactively
You can now view 6 of the 29738492734 different ways people try to manage worker nodes, namespaces, tenants, and accounts in Kubernetes clusters as part of, what might not be, my final post on Kubernetes.
This job is not hard because Kubernetes is hard. This job is hard because people are hard and people do not agree.
As a woman in engineering you get the benefit in your career of managing the onslaught of pain that comes from ever being assertive, laugh at the idea of even using the word “enforcement” when people ask you knowing that it can land you jobless, and simultaneously being told you need to influence more in rooms that have no one that looks like you and more often than not forget you exist when it counts.
That is what it means to be a woman in games engineering, but now add Kubernetes on top. I’m old enough now to be honest in life – I bought that with my time.
That said – I have loved almost every minute of it these last 4 years, and it’s a good time to put some of those architectural patterns into something thoughtful which is why I built this website in Claude Code and Claude Design. While it would have taken me only 4 hours total it took me much longer because I pay $20 a month and god it is expensive if you want to use Opus full time – I ran out of tokens in an hour every time I tried to work on it, which was whenever I had free time and wasn’t also building an interactive Disney Planner (more on that in the future).
You want K8s NaaS? I got you. Cell-based? I got you. For some reason everyone gets single-tenant clusters, each in their own accounts, that are all snowflake configurations with different deployment models? Okay I am sorry for you and me, but I got you covered, and I still support you and love you but hate myself.
I loved building this and sadly I could have kept going if being told to stop by Opus was not so annoying.
But the best parts of this is that it also has a fun comparison chart for cost verses pain and an entire recommendation of how AI Ops would actually integrate into these tenancy models. By the way the most expensive one is not single-tenant clusters.
It’s doing all of them in tandem.
So anyway, if you are the lucky recipient of a kubernetes business, do it. It’s worth it. But you also may be the lucky recipient of trying to AI Ops a kubernetes business, and that would be easy, if the diversification of tenancy types wasn’t so hard and had already been solved.
If you are the lucky recipient of a business that didn’t start in Kubernetes years ago, isn’t enterprise, and is just now moving from EC2 to K8s, wow you have such a lucky opportunity. Believe me, I get it. And I will always support you.
I don’t know what my next blog is going to be. A reasonable person would take a break and maybe write about rituals, but timing makes me think by the time I sit down to write again I won’t be able to predict and that’s okay. I’ve learned to lean into the signals of the universe and be ever so thankful for the wisdom and voice it has given me.
Rest in Peace, Matt Trescot – I wish you could have read this one. Thank you for always being there for me.
