Avatar of z00b
CTO at CircleCI ·

We use MongoDB as our primary #datastore. Mongo's approach to replica sets enables some fantastic patterns for operations like maintenance, backups, and #ETL.

As we pull #microservices from our #monolith, we are taking the opportunity to build them with their own datastores using PostgreSQL. We also use Redis to cache data we’d never store permanently, and to rate-limit our requests to partners’ APIs (like GitHub).

When we’re dealing with large blobs of immutable data (logs, artifacts, and test results), we store them in Amazon S3. We handle any side-effects of S3’s eventual consistency model within our own code. This ensures that we deal with user requests correctly while writes are in process.

Update: How CircleCI Processes Over 30 Million Builds Per Month - CircleCI Tech Stack (stackshare.io)
22 upvotes·1 comment·166.9K views
Manohar Sitapati
Manohar Sitapati
August 7th 2019 at 2:36pm

Very cool stack!

Avatar of Robert Zuber

Robert Zuber

CTO at CircleCI