Lorenzo's Blog About me

Backpack: helm charts but for Hashicorp Nomad

ATTENTION!

This article is 3 years old, it may be outdated or not relevant anymore! Comments and reactions might have been lost.

On  - Reading Time: 3 Minutes,

I recently decided to replace my k8s home cluster with a Hashicorp Nomad cluster on my 3 Raspberry Pis. When I was using it, I found myself writing a significant amount of HCL files, and I have been missing Helm’s simplicity.

As an SRE, I have been working with Kubernetes for a while now. During this time, I had to install, configure, and distribute some apps using Helm. For me, it is a de-facto standard when it comes to deploy and maintain apps that otherwise would require a way bigger list of YAML files. Can we do the same for Nomad Jobs’s definitions?

Go Gophers at work
Go Gophers at work

Yes, we can! For pure fun and during my spare time, I made Backpack for Nomad! 😀 I decided to keep things as similar as possible to Helm but improve on it too, all based on my personal experience.

# Please check the README.md as this might be outdated! 😉
go get -v gitlab.com/Qm64/backpack
cd $GOPATH/src/gitlab.com/Qm64/backpack/
make install
# Check that it works:
backpack help

Hint: I left some TL;DR in the README.md to quick start Nomad and test out backpack! 😉 Also note, that this article is written on version 0.1.0… things might change.

Backpack is like Helm

When I started writing the code of Backpack, I had to keep in mind that this is for myself or people like me… I am lazy 😅 therefore, I don’t want to learn a new templating system: I would find annoying to have to learn a new way when I have spent the past 5 years writing Helm charts! Instead, I decided to keep some things similar:

  • Backpack Uses the same Go templates and extra functions1
  • The default configuration/values is a YAML file2
  • All done with a single Go binary
  • Multiple workloads can be packed together in a Pack3
  • It is easy to create, test, and deploy new packs4

To get started you can create a boilerplate directory structure:

backpack create my-app

This command just created a directory my-app-0.1.0 with values.yaml containing the default values passed to the Go templates.

A Backpack is not just YAML

But there were a few things that I was not happy about and that I wanted to change. My big personal frustration was related to documentation. Often when there is a new version of a helm chart, the documentation for specific variables is lost, not synced, or hard to understand. 🙄 I often found myself deploying DataDog agents, but the version I was using was not using the same values defined in the documentation.

To solve this issue, I have decided to package the documentation and the templates, and the values altogether. This change makes sure that the version I am reading is the same I am configuring and deploying.

backpack unpack docs https://backpack.qm64.tech/examples/redis-6.0.0.backpack

This command just created a directory redis-6.0.0 containing only the Markdown files for the documentation. 😎 No more googling, or searching on github commits history. It is right there, next to the code!

Another feature that I wanted to improve on is that Helm chart focuses mostly on Kubernetes as a Docker-first platform. In Hashicorp Nomad, this is slightly different. A Job doesn’t have to be necessarily a docker container! It can be a binary downloaded on the fly5. An app that would support this is FabioLB: you can select the Job driver to be a docker container or a chroot-ed binary6. I already wrote a pack for it! 😉

# Deploy FabioLB using Docker driver for Nomad
echo "driver: docker" > values.yaml
backpack run https://backpack.qm64.tech/examples/fabiolb-1.5.14.backpack -v values.yaml
# Now Nomad will download the right docker image and start Fabio from there:
nomad ui fabiolb

# Switch to raw_exec (use exec for chroot, check driver docs)
echo "driver: raw_exec" > values.yaml
backpack run https://backpack.qm64.tech/examples/fabiolb-1.5.14.backpack -v values.yaml
# Now Nomad will download the right binaries (CPU and Platform) and start Fabio:
nomad ui fabiolb

Future plans for the project

This is just a starting point. I made this for myself, but I other poeple could find it useful as I do! 🤞

There are a few things I want to work on but had little time. I want to implement the packages and repository with IPFS in mind to avoid packages disappearing. I wish Backpack was smarter and capable of reading the cluster configuration to pick up automatically things like the job drive to use for the template, or like the service tags to implement to configure traefik or fabio-lb. I would love to pack multiple containers, apps, and tools, solve issues like dependency management and more…

I will do that later

I will do that later

For now, I will work on it in my spare time. I have released the source code on GitLab, and if you want to help feel free to ask a question on the Matrix chatroom or leave a comment to this post!

  1. I am using go templates with Sprig as Helm charts. ↩︎

  2. YAML might not be the most loved format, I do believe I can adjust the code to support multiple human-readable formats 🤔 ↩︎

  3. a Pack is the equivalent of an “Helm Chart”, open for suggestion if the name is not following the travelling theme 😜 ↩︎

  4. I think the core is to make it easy: backpack help might do the trick! ↩︎

  5. Nomad can dowload the right binaries, with the right CPU architecture (ex: amd64, armhf, arm64…) of the machine and the right OS (macOS, Linux…). All is done with variables: pure magic 😍 ↩︎

  6. This is still a work-in-progress feature. I just need to manually implement it in every template for now. Please check the documentation of the pack to know what drivers are supported! ↩︎