Stitch is a wrapper around a Cassandra database. It has a web application that provides read-access to the counts through an HTTP API. The counts are written to Cassandra in two distinct ways, and it's possible to use either or both of them:

• Real-time: For real-time updates, Stitch has a processor application that handles a stream of events coming from a broker and increments the appropriate counts in Cassandra.

• Batch: The batch part is a MapReduce job running on Hadoop that reads event logs, calculates the overall totals, and bulk loads this into Cassandra.

The final output is inserted into HBase to serve the experiment dashboard. We also load the output data to Redshift for ad-hoc analysis. For real-time experiment data processing, we use Storm to tail Kafka and process data in real-time and insert metrics into MySQL, so we could identify group allocation problems and send out real-time alerts and metrics.

Cassandra is our data management workhorse. It handles all our key-value services, supports time-series data storage and retrieval, securely stores all our audit trails, and backs our Datomic database.

While we experimented with Cassandra in the past, we are no longer using it. It is, however, open for consideration in future projects.

We are using Cassandra in a few of our apps. One of them is as a count service application to track the number of shares, clicks.. etc

Datastax interview

• Raw storage engine
• Replication
• Fault-tolerance

Range scan and HDFS Buffering system

Primary datastore