At uSwitch we wanted a way to load balance between our multiple Kubernetes clusters in AWS to give us added redundancy. We already had ingresses defined for all our applications so we wanted to build on top of that, instead of creating a new system that would require our various teams to change code/config etc.

Envoy seemed to tick a lot of boxes:

• Loadbalancing capabilities right out of the box: health checks, circuit breaking, retries etc.
• Tracing and prometheus metrics support
• Lightweight
• Good community support

This was all good but what really sold us was the api that supported dynamic configuration. This would allow us to dynamically configure envoy to route to ingresses and clusters as they were created or destroyed.

To do this we built a tool called Yggdrasil using their Go sdk. Yggdrasil effectively just creates envoy configuration from Kubernetes ingress objects, so you point Yggdrasil at your kube clusters, it generates config from the ingresses and then envoy can loadbalance between your clusters for you. This is all done dynamically so as soon as new ingress is created the envoy nodes get updated with the new config. Importantly this all worked with what we already had, no need to create new config for every application, we just put this on top of it.