Alternatives to Clickhouse logo

Alternatives to Clickhouse

Cassandra, Elasticsearch, MySQL, InfluxDB, and Druid are the most popular alternatives and competitors to Clickhouse.
187
233
+ 1
56

What is Clickhouse and what are its top alternatives?

It allows analysis of data that is updated in real time. It offers instant results in most cases: the data is processed faster than it takes to create a query.
Clickhouse is a tool in the Databases category of a tech stack.

Top Alternatives to Clickhouse

  • Cassandra

    Cassandra

    Partitioning means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster. Row store means that like relational databases, Cassandra organizes data by rows and columns. The Cassandra Query Language (CQL) is a close relative of SQL. ...

  • Elasticsearch

    Elasticsearch

    Elasticsearch is a distributed, RESTful search and analytics engine capable of storing data and searching it in near real time. Elasticsearch, Kibana, Beats and Logstash are the Elastic Stack (sometimes called the ELK Stack). ...

  • MySQL

    MySQL

    The MySQL software delivers a very fast, multi-threaded, multi-user, and robust SQL (Structured Query Language) database server. MySQL Server is intended for mission-critical, heavy-load production systems as well as for embedding into mass-deployed software. ...

  • InfluxDB

    InfluxDB

    InfluxDB is a scalable datastore for metrics, events, and real-time analytics. It has a built-in HTTP API so you don't have to write any server side code to get up and running. InfluxDB is designed to be scalable, simple to install and manage, and fast to get data in and out. ...

  • Druid

    Druid

    Druid is a distributed, column-oriented, real-time analytics data store that is commonly used to power exploratory dashboards in multi-tenant environments. Druid excels as a data warehousing solution for fast aggregate queries on petabyte sized data sets. Druid supports a variety of flexible filters, exact calculations, approximate algorithms, and other useful calculations. ...

  • MongoDB

    MongoDB

    MongoDB stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema. MongoDB was also designed for high availability and scalability, with built-in replication and auto-sharding. ...

  • Vertica

    Vertica

    It provides a best-in-class, unified analytics platform that will forever be independent from underlying infrastructure. ...

  • Snowflake

    Snowflake

    Snowflake eliminates the administration and management demands of traditional data warehouses and big data platforms. Snowflake is a true data warehouse as a service running on Amazon Web Services (AWS)—no infrastructure to manage and no knobs to turn. ...

Clickhouse alternatives & related posts

Cassandra logo

Cassandra

3K
2.9K
463
A partitioned row store. Rows are organized into tables with a required primary key.
3K
2.9K
+ 1
463
PROS OF CASSANDRA
  • 107
    Distributed
  • 90
    High performance
  • 77
    High availability
  • 71
    Easy scalability
  • 50
    Replication
  • 25
    Reliable
  • 24
    Multi datacenter deployments
  • 6
    Schema optional
  • 6
    OLTP
  • 5
    Open source
  • 2
    Workload separation (via MDC)
CONS OF CASSANDRA
  • 1
    Reliability of replication
  • 1
    Updates

related Cassandra posts

Thierry Schellenbach
Shared insights
on
RedisRedisCassandraCassandraRocksDBRocksDB
at

1.0 of Stream leveraged Cassandra for storing the feed. Cassandra is a common choice for building feeds. Instagram, for instance started, out with Redis but eventually switched to Cassandra to handle their rapid usage growth. Cassandra can handle write heavy workloads very efficiently.

Cassandra is a great tool that allows you to scale write capacity simply by adding more nodes, though it is also very complex. This complexity made it hard to diagnose performance fluctuations. Even though we had years of experience with running Cassandra, it still felt like a bit of a black box. When building Stream 2.0 we decided to go for a different approach and build Keevo. Keevo is our in-house key-value store built upon RocksDB, gRPC and Raft.

RocksDB is a highly performant embeddable database library developed and maintained by Facebook’s data engineering team. RocksDB started as a fork of Google’s LevelDB that introduced several performance improvements for SSD. Nowadays RocksDB is a project on its own and is under active development. It is written in C++ and it’s fast. Have a look at how this benchmark handles 7 million QPS. In terms of technology it’s much more simple than Cassandra.

This translates into reduced maintenance overhead, improved performance and, most importantly, more consistent performance. It’s interesting to note that LinkedIn also uses RocksDB for their feed.

#InMemoryDatabases #DataStores #Databases

See more
Umair Iftikhar
Technical Architect at Vappar · | 3 upvotes · 12.7K views

Developing a solution that collects Telemetry Data from different devices, nearly 1000 devices minimum and maximum 12000. Each device is sending 2 packets in 1 second. This is time-series data, and this data definition and different reports are saved on PostgreSQL. Like Building information, maintenance records, etc. I want to know about the best solution. This data is required for Math and ML to run different algorithms. Also, data is raw without definitions and information stored in PostgreSQL. Initially, I went with TimescaleDB due to PostgreSQL support, but to increase in sites, I started facing many issues with timescale DB in terms of flexibility of storing data.

My major requirement is also the replication of the database for reporting and different purposes. You may also suggest other options other than Druid and Cassandra. But an open source solution is appreciated.

See more
Elasticsearch logo

Elasticsearch

22.2K
16.3K
1.6K
Open Source, Distributed, RESTful Search Engine
22.2K
16.3K
+ 1
1.6K
PROS OF ELASTICSEARCH
  • 320
    Powerful api
  • 310
    Great search engine
  • 230
    Open source
  • 213
    Restful
  • 200
    Near real-time search
  • 96
    Free
  • 83
    Search everything
  • 54
    Easy to get started
  • 45
    Analytics
  • 26
    Distributed
  • 6
    Fast search
  • 5
    More than a search engine
  • 3
    Awesome, great tool
  • 3
    Easy to scale
  • 3
    Great docs
  • 2
    Nosql DB
  • 2
    Great piece of software
  • 2
    Document Store
  • 2
    Great customer support
  • 2
    Intuitive API
  • 2
    Fast
  • 2
    Easy setup
  • 2
    Highly Available
  • 1
    Not stable
  • 1
    Reliable
  • 1
    Github
  • 1
    Elaticsearch
  • 1
    Actively developing
  • 1
    Responsive maintainers on GitHub
  • 1
    Ecosystem
  • 1
    Scalability
  • 1
    Potato
  • 0
    Community
  • 0
    Easy to get hot data
CONS OF ELASTICSEARCH
  • 6
    Diffecult to get started
  • 5
    Resource hungry
  • 4
    Expensive
  • 3
    Hard to keep stable at large scale

related Elasticsearch posts

Tim Abbott

We've been using PostgreSQL since the very early days of Zulip, but we actually didn't use it from the beginning. Zulip started out as a MySQL project back in 2012, because we'd heard it was a good choice for a startup with a wide community. However, we found that even though we were using the Django ORM for most of our database access, we spent a lot of time fighting with MySQL. Issues ranged from bad collation defaults, to bad query plans which required a lot of manual query tweaks.

We ended up getting so frustrated that we tried out PostgresQL, and the results were fantastic. We didn't have to do any real customization (just some tuning settings for how big a server we had), and all of our most important queries were faster out of the box. As a result, we were able to delete a bunch of custom queries escaping the ORM that we'd written to make the MySQL query planner happy (because postgres just did the right thing automatically).

And then after that, we've just gotten a ton of value out of postgres. We use its excellent built-in full-text search, which has helped us avoid needing to bring in a tool like Elasticsearch, and we've really enjoyed features like its partial indexes, which saved us a lot of work adding unnecessary extra tables to get good performance for things like our "unread messages" and "starred messages" indexes.

I can't recommend it highly enough.

See more
Tymoteusz Paul
Devops guy at X20X Development LTD · | 21 upvotes · 4M views

Often enough I have to explain my way of going about setting up a CI/CD pipeline with multiple deployment platforms. Since I am a bit tired of yapping the same every single time, I've decided to write it up and share with the world this way, and send people to read it instead ;). I will explain it on "live-example" of how the Rome got built, basing that current methodology exists only of readme.md and wishes of good luck (as it usually is ;)).

It always starts with an app, whatever it may be and reading the readmes available while Vagrant and VirtualBox is installing and updating. Following that is the first hurdle to go over - convert all the instruction/scripts into Ansible playbook(s), and only stopping when doing a clear vagrant up or vagrant reload we will have a fully working environment. As our Vagrant environment is now functional, it's time to break it! This is the moment to look for how things can be done better (too rigid/too lose versioning? Sloppy environment setup?) and replace them with the right way to do stuff, one that won't bite us in the backside. This is the point, and the best opportunity, to upcycle the existing way of doing dev environment to produce a proper, production-grade product.

I should probably digress here for a moment and explain why. I firmly believe that the way you deploy production is the same way you should deploy develop, shy of few debugging-friendly setting. This way you avoid the discrepancy between how production work vs how development works, which almost always causes major pains in the back of the neck, and with use of proper tools should mean no more work for the developers. That's why we start with Vagrant as developer boxes should be as easy as vagrant up, but the meat of our product lies in Ansible which will do meat of the work and can be applied to almost anything: AWS, bare metal, docker, LXC, in open net, behind vpn - you name it.

We must also give proper consideration to monitoring and logging hoovering at this point. My generic answer here is to grab Elasticsearch, Kibana, and Logstash. While for different use cases there may be better solutions, this one is well battle-tested, performs reasonably and is very easy to scale both vertically (within some limits) and horizontally. Logstash rules are easy to write and are well supported in maintenance through Ansible, which as I've mentioned earlier, are at the very core of things, and creating triggers/reports and alerts based on Elastic and Kibana is generally a breeze, including some quite complex aggregations.

If we are happy with the state of the Ansible it's time to move on and put all those roles and playbooks to work. Namely, we need something to manage our CI/CD pipelines. For me, the choice is obvious: TeamCity. It's modern, robust and unlike most of the light-weight alternatives, it's transparent. What I mean by that is that it doesn't tell you how to do things, doesn't limit your ways to deploy, or test, or package for that matter. Instead, it provides a developer-friendly and rich playground for your pipelines. You can do most the same with Jenkins, but it has a quite dated look and feel to it, while also missing some key functionality that must be brought in via plugins (like quality REST API which comes built-in with TeamCity). It also comes with all the common-handy plugins like Slack or Apache Maven integration.

The exact flow between CI and CD varies too greatly from one application to another to describe, so I will outline a few rules that guide me in it: 1. Make build steps as small as possible. This way when something breaks, we know exactly where, without needing to dig and root around. 2. All security credentials besides development environment must be sources from individual Vault instances. Keys to those containers should exist only on the CI/CD box and accessible by a few people (the less the better). This is pretty self-explanatory, as anything besides dev may contain sensitive data and, at times, be public-facing. Because of that appropriate security must be present. TeamCity shines in this department with excellent secrets-management. 3. Every part of the build chain shall consume and produce artifacts. If it creates nothing, it likely shouldn't be its own build. This way if any issue shows up with any environment or version, all developer has to do it is grab appropriate artifacts to reproduce the issue locally. 4. Deployment builds should be directly tied to specific Git branches/tags. This enables much easier tracking of what caused an issue, including automated identifying and tagging the author (nothing like automated regression testing!).

Speaking of deployments, I generally try to keep it simple but also with a close eye on the wallet. Because of that, I am more than happy with AWS or another cloud provider, but also constantly peeking at the loads and do we get the value of what we are paying for. Often enough the pattern of use is not constantly erratic, but rather has a firm baseline which could be migrated away from the cloud and into bare metal boxes. That is another part where this approach strongly triumphs over the common Docker and CircleCI setup, where you are very much tied in to use cloud providers and getting out is expensive. Here to embrace bare-metal hosting all you need is a help of some container-based self-hosting software, my personal preference is with Proxmox and LXC. Following that all you must write are ansible scripts to manage hardware of Proxmox, similar way as you do for Amazon EC2 (ansible supports both greatly) and you are good to go. One does not exclude another, quite the opposite, as they can live in great synergy and cut your costs dramatically (the heavier your base load, the bigger the savings) while providing production-grade resiliency.

See more
MySQL logo

MySQL

68.3K
52.7K
3.7K
The world's most popular open source database
68.3K
52.7K
+ 1
3.7K
PROS OF MYSQL
  • 789
    Sql
  • 674
    Free
  • 557
    Easy
  • 527
    Widely used
  • 485
    Open source
  • 180
    High availability
  • 158
    Cross-platform support
  • 103
    Great community
  • 77
    Secure
  • 75
    Full-text indexing and searching
  • 25
    Fast, open, available
  • 14
    SSL support
  • 13
    Robust
  • 13
    Reliable
  • 8
    Enterprise Version
  • 7
    Easy to set up on all platforms
  • 1
    Easy, light, scalable
  • 1
    Relational database
  • 1
    NoSQL access to JSON data type
  • 1
    Sequel Pro (best SQL GUI)
  • 1
    Replica Support
CONS OF MYSQL
  • 13
    Owned by a company with their own agenda
  • 1
    Can't roll back schema changes

related MySQL posts

Tim Abbott

We've been using PostgreSQL since the very early days of Zulip, but we actually didn't use it from the beginning. Zulip started out as a MySQL project back in 2012, because we'd heard it was a good choice for a startup with a wide community. However, we found that even though we were using the Django ORM for most of our database access, we spent a lot of time fighting with MySQL. Issues ranged from bad collation defaults, to bad query plans which required a lot of manual query tweaks.

We ended up getting so frustrated that we tried out PostgresQL, and the results were fantastic. We didn't have to do any real customization (just some tuning settings for how big a server we had), and all of our most important queries were faster out of the box. As a result, we were able to delete a bunch of custom queries escaping the ORM that we'd written to make the MySQL query planner happy (because postgres just did the right thing automatically).

And then after that, we've just gotten a ton of value out of postgres. We use its excellent built-in full-text search, which has helped us avoid needing to bring in a tool like Elasticsearch, and we've really enjoyed features like its partial indexes, which saved us a lot of work adding unnecessary extra tables to get good performance for things like our "unread messages" and "starred messages" indexes.

I can't recommend it highly enough.

See more
Conor Myhrvold
Tech Brand Mgr, Office of CTO at Uber · | 20 upvotes · 916K views

Our most popular (& controversial!) article to date on the Uber Engineering blog in 3+ yrs. Why we moved from PostgreSQL to MySQL. In essence, it was due to a variety of limitations of Postgres at the time. Fun fact -- earlier in Uber's history we'd actually moved from MySQL to Postgres before switching back for good, & though we published the article in Summer 2016 we haven't looked back since:

The early architecture of Uber consisted of a monolithic backend application written in Python that used Postgres for data persistence. Since that time, the architecture of Uber has changed significantly, to a model of microservices and new data platforms. Specifically, in many of the cases where we previously used Postgres, we now use Schemaless, a novel database sharding layer built on top of MySQL (https://eng.uber.com/schemaless-part-one/). In this article, we’ll explore some of the drawbacks we found with Postgres and explain the decision to build Schemaless and other backend services on top of MySQL:

https://eng.uber.com/mysql-migration/

See more
InfluxDB logo

InfluxDB

784
788
157
An open-source distributed time series database with no external dependencies
784
788
+ 1
157
PROS OF INFLUXDB
  • 46
    Time-series data analysis
  • 27
    Easy setup, no dependencies
  • 24
    Fast, scalable & open source
  • 21
    Open source
  • 18
    Real-time analytics
  • 6
    Continuous Query support
  • 5
    Easy Query Language
  • 4
    Out-of-the-box, automatic Retention Policy
  • 4
    HTTP API
  • 1
    Offers Enterprise version
  • 1
    Free Open Source version
CONS OF INFLUXDB
  • 4
    Instability
  • 1
    HA or Clustering is only in paid version

related InfluxDB posts

Druid logo

Druid

264
552
26
Fast column-oriented distributed data store
264
552
+ 1
26
PROS OF DRUID
  • 13
    Real Time Aggregations
  • 4
    OLAP
  • 4
    Batch and Real-Time Ingestion
  • 2
    Combining stream and historical analytics
  • 2
    OLAP + OLTP
  • 1
    OLTP
CONS OF DRUID
  • 2
    Limited sql support
  • 1
    Complexity
  • 1
    Joins are not supported well

related Druid posts

Umair Iftikhar
Technical Architect at Vappar · | 3 upvotes · 12.7K views

Developing a solution that collects Telemetry Data from different devices, nearly 1000 devices minimum and maximum 12000. Each device is sending 2 packets in 1 second. This is time-series data, and this data definition and different reports are saved on PostgreSQL. Like Building information, maintenance records, etc. I want to know about the best solution. This data is required for Math and ML to run different algorithms. Also, data is raw without definitions and information stored in PostgreSQL. Initially, I went with TimescaleDB due to PostgreSQL support, but to increase in sites, I started facing many issues with timescale DB in terms of flexibility of storing data.

My major requirement is also the replication of the database for reporting and different purposes. You may also suggest other options other than Druid and Cassandra. But an open source solution is appreciated.

See more
MongoDB logo

MongoDB

51.6K
41.3K
4K
The database for giant ideas
51.6K
41.3K
+ 1
4K
PROS OF MONGODB
  • 822
    Document-oriented storage
  • 585
    No sql
  • 544
    Ease of use
  • 462
    Fast
  • 404
    High performance
  • 251
    Free
  • 212
    Open source
  • 177
    Flexible
  • 139
    Replication & high availability
  • 107
    Easy to maintain
  • 39
    Querying
  • 35
    Easy scalability
  • 34
    Auto-sharding
  • 33
    High availability
  • 29
    Map/reduce
  • 26
    Document database
  • 24
    Easy setup
  • 24
    Full index support
  • 15
    Reliable
  • 14
    Fast in-place updates
  • 13
    Agile programming, flexible, fast
  • 11
    No database migrations
  • 7
    Enterprise
  • 7
    Easy integration with Node.Js
  • 5
    Enterprise Support
  • 4
    Great NoSQL DB
  • 3
    Aggregation Framework
  • 3
    Drivers support is good
  • 3
    Support for many languages through different drivers
  • 2
    Schemaless
  • 2
    Managed service
  • 2
    Easy to Scale
  • 2
    Fast
  • 2
    Awesome
  • 1
    Consistent
CONS OF MONGODB
  • 5
    Very slowly for connected models that require joins
  • 3
    Not acid compliant
  • 1
    Proprietary query language

related MongoDB posts

Jeyabalaji Subramanian

Recently we were looking at a few robust and cost-effective ways of replicating the data that resides in our production MongoDB to a PostgreSQL database for data warehousing and business intelligence.

We set ourselves the following criteria for the optimal tool that would do this job: - The data replication must be near real-time, yet it should NOT impact the production database - The data replication must be horizontally scalable (based on the load), asynchronous & crash-resilient

Based on the above criteria, we selected the following tools to perform the end to end data replication:

We chose MongoDB Stitch for picking up the changes in the source database. It is the serverless platform from MongoDB. One of the services offered by MongoDB Stitch is Stitch Triggers. Using stitch triggers, you can execute a serverless function (in Node.js) in real time in response to changes in the database. When there are a lot of database changes, Stitch automatically "feeds forward" these changes through an asynchronous queue.

We chose Amazon SQS as the pipe / message backbone for communicating the changes from MongoDB to our own replication service. Interestingly enough, MongoDB stitch offers integration with AWS services.

In the Node.js function, we wrote minimal functionality to communicate the database changes (insert / update / delete / replace) to Amazon SQS.

Next we wrote a minimal micro-service in Python to listen to the message events on SQS, pickup the data payload & mirror the DB changes on to the target Data warehouse. We implemented source data to target data translation by modelling target table structures through SQLAlchemy . We deployed this micro-service as AWS Lambda with Zappa. With Zappa, deploying your services as event-driven & horizontally scalable Lambda service is dumb-easy.

In the end, we got to implement a highly scalable near realtime Change Data Replication service that "works" and deployed to production in a matter of few days!

See more
Robert Zuber

We use MongoDB as our primary #datastore. Mongo's approach to replica sets enables some fantastic patterns for operations like maintenance, backups, and #ETL.

As we pull #microservices from our #monolith, we are taking the opportunity to build them with their own datastores using PostgreSQL. We also use Redis to cache data we’d never store permanently, and to rate-limit our requests to partners’ APIs (like GitHub).

When we’re dealing with large blobs of immutable data (logs, artifacts, and test results), we store them in Amazon S3. We handle any side-effects of S3’s eventual consistency model within our own code. This ensures that we deal with user requests correctly while writes are in process.

See more
Vertica logo

Vertica

61
65
13
Storage platform designed to handle large volumes of data
61
65
+ 1
13
PROS OF VERTICA
  • 1
    Shared nothing or shared everything architecture
  • 1
    Offers users the freedom to choose deployment mode
  • 1
    Flexible architecture suits nearly any project
  • 1
    End-to-End ML Workflow Support
  • 1
    All You Need for IoT, Clickstream or Geospatial
  • 1
    Freedom from Underlying Storage
  • 1
    Pre-Aggregation for Cubes (LAPS)
  • 1
    Automatic Data Marts (Flatten Tables)
  • 1
    Near-Real-Time Analytics in pure Column Store
  • 1
    Fully automated Database Designer tool
  • 1
    Query-Optimized Storage
  • 1
    Vertica is the only product which offers partition prun
  • 1
    Partition pruning and predicate push down on Parquet
CONS OF VERTICA
    Be the first to leave a con

    related Vertica posts

    Snowflake logo

    Snowflake

    470
    564
    6
    The data warehouse built for the cloud
    470
    564
    + 1
    6
    PROS OF SNOWFLAKE
    • 2
      Good Performance
    • 1
      Public and Private Data Sharing
    • 1
      Multicloud
    • 1
      Great Documentation
    • 1
      Serverless
    CONS OF SNOWFLAKE
      Be the first to leave a con

      related Snowflake posts

      Shared insights
      on
      Google BigQueryGoogle BigQuerySnowflakeSnowflake

      I use Google BigQuery because it makes is super easy to query and store data for analytics workloads. If you're using GCP, you're likely using BigQuery. However, running data viz tools directly connected to BigQuery will run pretty slow. They recently announced BI Engine which will hopefully compete well against big players like Snowflake when it comes to concurrency.

      What's nice too is that it has SQL-based ML tools, and it has great GIS support!

      See more