Need advice about which tool to choose?Ask the StackShare community!
HBase vs Scylla: What are the differences?
Introduction
HBase and Scylla are both distributed, highly scalable NoSQL databases designed for handling big data workloads. While they share some similarities, there are key differences that distinguish them from each other.
Data Model and Query Language: One significant difference between HBase and Scylla is their data model and query language. HBase follows a columnar data model with a hierarchical structure similar to Google's Bigtable, while Scylla is based on Cassandra's row-oriented data model. HBase uses the Hadoop ecosystem's query language HQL (HBase Query Language), whereas Scylla uses CQL (Cassandra Query Language). This disparity in data models and query languages affects how developers interact with and manipulate the data in each database.
Consistency and Availability: Another important distinction lies in their consistency and availability models. HBase prioritizes strong consistency, ensuring that read and write operations return the most up-to-date data and guaranteeing data integrity at the expense of potential latency and throughput reductions during periods of high load or failure. On the other hand, Scylla employs eventual consistency by default, which allows for higher availability and performance but introduces the possibility of stale reads and inconsistent data.
Storage Model: HBase and Scylla differ in their storage models as well. HBase utilizes Hadoop's HDFS (Hadoop Distributed File System) for storing data, while Scylla employs its own storage engine, Seastar. The use of different storage systems can impact factors such as data durability, fault tolerance, and performance capabilities.
Scalability: Both HBase and Scylla are designed for scalability, but they employ different scaling mechanisms. HBase relies on the horizontal scaling approach provided by Hadoop's HDFS and HBase RegionServers, distributing data across multiple nodes. Scylla, on the other hand, leverages Cassandra's peer-to-peer architecture, allowing it to horizontally scale by adding more nodes to the cluster. Each database's scalability mechanisms come with their own set of advantages and considerations, depending on the specific use case and workload requirements.
Community Support and Maturity: HBase has been in development and widely deployed for a longer time compared to Scylla, giving it a more mature and established community. It has a larger user base, more extensive documentation, and a wider range of community-driven extensions and tools. However, Scylla benefits from the active Cassandra community and inherits its ecosystem, which includes a variety of plugins, connectors, and libraries.
Data Compression: HBase and Scylla employ different data compression techniques. HBase supports multiple compression algorithms such as Snappy, Gzip, and LZO, allowing users to choose the most suitable compression method based on their specific requirements. On the other hand, Scylla utilizes LZ4 compression, delivering higher compression and decompression speeds compared to other algorithms. The choice of compression technique can influence storage utilization, read and write performance, and CPU usage.
In summary, HBase and Scylla differ in their data models and query languages, consistency and availability models, storage models, scaling mechanisms, community support and maturity, and data compression techniques. These differences play a crucial role in determining which database is the best fit for a particular use case and workload requirements.
The problem I have is - we need to process & change(update/insert) 55M Data every 2 min and this updated data to be available for Rest API for Filtering / Selection. Response time for Rest API should be less than 1 sec.
The most important factors for me are processing and storing time of 2 min. There need to be 2 views of Data One is for Selection & 2. Changed data.
Scylla can handle 1M/s events with a simple data model quite easily. The api to query is CQL, we have REST api but that's for control/monitoring
Cassandra is quite capable of the task, in a highly available way, given appropriate scaling of the system. Remember that updates are only inserts, and that efficient retrieval is only by key (which can be a complex key). Talking of keys, make sure that the keys are well distributed.
By 55M do you mean 55 million entity changes per 2 minutes? It is relatively high, means almost 460k per second. If I had to choose between Scylla or Cassandra, I would opt for Scylla as it is promising better performance for simple operations. However, maybe it would be worth to consider yet another alternative technology. Take into consideration required consistency, reliability and high availability and you may realize that there are more suitable once. Rest API should not be the main driver, because you can always develop the API yourself, if not supported by given technology.
i love syclla for pet projects however it's license which is based on server model is an issue. thus i recommend cassandra
The Gentlent Tech Team made lots of updates within the past year. The biggest one being our database:
We decided to migrate our #PostgreSQL -based database systems to a custom implementation of #Cassandra . This allows us to integrate our product data perfectly in a system that just makes sense. High availability and scalability are supported out of the box.
Pros of HBase
- Performance9
- OLTP5
- Fast Point Queries1
Pros of ScyllaDB
- Replication2
- Fewer nodes1
- Distributed1
- Scale up1
- High availability1
- Written in C++1
- High performance1