Cassandra

We migrated most of our APIs last year from using our self managed Cassandra cluster to a mix of Amazon DynamoDB and Amazon RDS for Aurora. This has reduced the operational overhead for our team and greatly improved the overall reliability of our service. The new dynamic capacity in DynamoDB has been super helpful for handling bursty traffic.

Onedot is building an automated data preparation service using probabilistic and statistical methods including artificial intelligence (AI). From the beginning, having a stable foundation while at the same time being able to iterate quickly was very important to us. Due to the nature of compute workloads we face, the decision for a functional programming paradigm and a scalable cluster model was a no-brainer. We started playing with Apache Spark very early on, when the platform was still in its infancy. As a storage backend, we first used Cassandra, but found out that it was not the optimal choice for our workloads (lots of rather smallish datasets, data pipelines with considerable complexity, etc.). In the end, we migrated dataset storage to Amazon S3 which proved to be much more adequate to our case. In the frontend, we bet on more traditional frameworks like React/Redux.js, Blueprint and a number of common npm packages of our universe. Because of the very positive experience with Scala (in particular the ability to write things very expressively, use immutability across the board, etc.) we settled with TypeScript in the frontend. In our opinion, a very good decision. Nowadays, transpiling is a common thing, so we thought why not introduce the same type-safety and mathematical rigour to the user interface?

I use Cassandra because scales horizontally at ease. Provides availability and partition tolerance. Cassandra can be used only if we know upfront all the read patterns on the data.

Stitch is a wrapper around a Cassandra database. It has a web application that provides read-access to the counts through an HTTP API. The counts are written to Cassandra in two distinct ways, and it's possible to use either or both of them:

• Real-time: For real-time updates, Stitch has a processor application that handles a stream of events coming from a broker and increments the appropriate counts in Cassandra.

• Batch: The batch part is a MapReduce job running on Hadoop that reads event logs, calculates the overall totals, and bulk loads this into Cassandra. Cassandra

React AngularJS jQuery

Laravel Zend Framework

MySQL MongoDB Cassandra

Docker

Linux

While we experimented with Cassandra in the past, we are no longer using it. It is, however, open for consideration in future projects. Cassandra