Mid-2019, we started a search for a better solution for all of our analytical needs. Our existing analytical infrastructure had started to fall apart; it was incapable of keeping up with the rapid growth of the data we were collecting and analyzing at the time. Trino (previously known as Presto SQL) became our prime candidate since it had a feature set we needed; was actively maintained by and active community (and also commercial companies like Facebook and Starburst); had seamless integration with our BI tools; and last but not least, was successfully used by the biggest companies in tech — Facebook, Uber, Netflix, AirBnB, and many others. It remains at the heart of analytics at Flo, helping us to make truly data-driven decisions ever since.

One particular aspect of Trino that distinguishes it from the competition is its ability to integrate a large variety of data sources thanks to its Connector API. With this feature, you can do things like expose data from Kafka topics as Trino tables or query your favorite SQL/NoSQL database. Trino's performance is also great because it allows us to process workloads requiring relatively low latency and process long-running queries effectively.

A bit of trivia: Since all of our infrastructure is deployed in AWS, at the time, it was tempting for us to deploy Presto as a part of Amazon EMR; however, we went with a different approach. There were two flavors of Presto at the time (and still are): the original Facebook version and a fork named PrestoSQL (later rebranded as Trino due to copyright claims) that was created by the team of original presto authors who had departed from Facebook. After some extensive internal benchmarking, we chose the latter, since it has a better feature set and a much higher development pace. And since it wasn't available as a part of EMR, we ended up deploying PrestoSQL in the Kubernetes cluster, which allowed us to scale the cluster up and down better and faster depending on the current workload.

It's not a perfect tool, though — a fast development pace means that sometimes you find things broken, but that's OK).

Presto: SQL on Everything

Ivan Sharamet

Tools Used

Most Used

By Layer

Application & Data