MachineShop

Kafka was only introduced to our platform in August 2018 as a means to manage our data pipeline and to replace other messaging systems used to decouple various components in our system. Kafka provides the scale and storage we need to manage data for however many devices we might service. Additionally, Kafka has helped us lay the framework for improved and highly detailed statistics gathering and analysis.

MachineShop adopted Go early on thanks to its robust ability to target specific architectures during builds, a key component of our edge daemon. As of January 2018, we now also run Go for the vast majority of our backend services as well thanks to efforts to move away from Java-based systems that weren't container-friendly. Additionally, we operate on a monorepo+microservice structure that allows us to ship highly specialized services when needed while also maximizing code re-use.

nginx became part of our stack largely by virtue of the ingress-nginx plugin for Kubernetes. It's proved reliable and easy to work with and helped us bring down our costs by moving from AWS Elastic Load Balancing (ELB)-backed services to Kubernetes ingresses.

We migrated to Kubernetes around January of 2018 as a way to reduce operations costs and manage future scaling. This decision forced us into other improvements and better habits which have paid large dividends in product stability and developer productivity. Thanks largely to kops, choosing Kubernetes in particular was easy because deploying and configuring it was easy.