Before there was Helm, I wrote my own tool and framework in Ruby (called Matsuri) to generate and manage manifests. I still use it. However, the source of truth is still on the kube api server. I did write in functionality to diff what is in the git repo and what is in the source of truth. There is an additional abstraction that bundles together resources so that a single command can converge them all together. The underlying mechanism is still kubectl, and the code merely generates the manifests to send it to kubectl via stdin.
I did not think about a “delete”, and I usually don’t want it to be automatically deleted. But that’s a great idea for that convergence mechanism if I have something that explicitly adds a deleted resource line to make sure it stays deleted until otherwise specified.
The Ruby code can access anything Ruby to generate the manifests, so I have a Turing-complete language and I can use class inheritance, mixins, method overrides. I was also able to write a suite of helpers to take a raw yaml manifest from an upstream project, and use a functional programming style to transform the manifest to something else. If I ever have time, I’d write the ability to import templates from
Helm.
This was written for a one-person devops role, intended to be able to manage almost-similar things across different environments (dev, staging, prod), which works great for small, early stage startups, for our level of complexity. At this level, I don’t need it to track state.
When our team size grows, I’ll have to think up of some other patterns to make it easier for multiple people to work on this.
The other thing is that for much more complex things, I would just write operator code using Elixir, and bootstrap that into k8s with the Ruby framework.
> Before there was Helm, I wrote my own tool and framework in Ruby (called Matsuri) to generate and manage manifests.
I've built these kinds of things too in Python (or Starlark). It's an improvement over YAML, but I often feel the pain of dynamic typing.
> The Ruby code can access anything Ruby to generate the manifests, so I have a Turing-complete language and I can use class inheritance, mixins, method overrides.
I actually don't want any of these things in a dynamic configuration language. I kind of just want something that can evaluate expressions (including functions, objects, arrays, scalars, and pattern matching). If it's Turing complete that's fine but unnecessary. I explicitly don't want I/O (not in your list) or inheritance, mixins, method overrides, etc--unnecessary complexity.
Dhall probably comes the closest, but it has a tragic syntax and I'm not a good enough salesman to persuade real software teams to learn it. I've also heard good things about Cue, but my experiences playing with it have only been painful (steep learning curve, don't really see the upsides).
I don't really see this tooling as a dynamic configuration language so much as a manifest generator. That means I also have tooling for diffing, debugging, and converging manifests. If you are just focused on configuration, I can see why Turing-completeness will seem like unnessary complexity.
I use classes and mixins to be able to generate similar manifests with slight differences across different clusters or environments. I sometimes use imports (I/O) for manifests provided by an upstream (such as from AWS docs), do transforms to get manifest I want.
I modelled the design off of Chef, which will also declaritively define and converge a set of systems towards desired state. Well-known paths and conventions helps keep things organized.
I havn't had a problem with dynamic typing, but I have also been using Ruby in application development for over 10 years. You might see it as unneeded complexity, but I have been able to use the flexibility for years now. This tooling was designed for a team that uses Ruby as the primary language, and uses designs and idom that would be familiar for a Ruby dev team. It is definitely opinionated and I don't expect it to be universally useful for everyone.
I did not think about a “delete”, and I usually don’t want it to be automatically deleted. But that’s a great idea for that convergence mechanism if I have something that explicitly adds a deleted resource line to make sure it stays deleted until otherwise specified.
The Ruby code can access anything Ruby to generate the manifests, so I have a Turing-complete language and I can use class inheritance, mixins, method overrides. I was also able to write a suite of helpers to take a raw yaml manifest from an upstream project, and use a functional programming style to transform the manifest to something else. If I ever have time, I’d write the ability to import templates from Helm.
This was written for a one-person devops role, intended to be able to manage almost-similar things across different environments (dev, staging, prod), which works great for small, early stage startups, for our level of complexity. At this level, I don’t need it to track state.
When our team size grows, I’ll have to think up of some other patterns to make it easier for multiple people to work on this.
The other thing is that for much more complex things, I would just write operator code using Elixir, and bootstrap that into k8s with the Ruby framework.