The main reasons for using Go as a Kafka consumer at Deliveroo are (1) simplicity and reliabiliy of code (2) speed of execution.

For (1), Python works just as well and indeed, we do have some low-scale applications using Python as it tends to be more productive for smaller and/or shorter life projects. But Go remains upwards compatible since its inception while the 2->3 conversion in Python was no small matter, and even minor version conversions tend to require many dependency upgrades, which tend not to be synchronised. For (2) at high scale (think 1M transactions/5 minutes), Go is simpler to scale on less hardware. We also use Kotlin and Scala for this because, beyond pure manually written consumers, Go misses a real equivalent to Kafka Streams, as that only exists on the JVM.

Deliveroo

Company Members