Lykon

We decided to use python to write our ETLs and import them into metabase via a lambda. Before python we tried using Go, but overall go was way more verbose than Python when writing the ETLs. Go also had some issues managing memory when using the S3 upload manager library. This was a deal breaker for us that made us switch to Python.

In the end the solution was much cleaner and maintainable.

Go is our language of choice for most of our microservices. The simplicity and power of Go is what makes us love it. It's fun to write, has a very powerful standard library that we try to stick to. In case what we are looking for is not in the std lib then it has a huge community with tons of libraries to choose from.

Scalability is another important point that go thrives. Simple concurrency models that are easy to manage are very appealing to us.

Grafana is a great tool for data visualization. It integrates quite easily with Prometheus and many different other data sources. In our case, we automate the whole grafana cluster in our Kubernetes cluster, with helm charts. All grafana charts are also automatically loaded from each service in our infrastructure. This allows us to version dashbaords.

We chose helm as our package management for kubernetes applications because of mainly these topics:

1. Big community: In case we need help, there is always someone there
2. Active developement: always trying to improve the tool
3. Easy to learn: Using simple go templates we can have a very powerful package