Alternatives to HBase logo

Alternatives to HBase

Cassandra, MongoDB, Hadoop, Druid, and Couchbase are the most popular alternatives and competitors to HBase.
325
371
+ 1
14

What is HBase and what are its top alternatives?

Apache HBase is an open-source, distributed, versioned, column-oriented store modeled after Google' Bigtable: A Distributed Storage System for Structured Data by Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System, HBase provides Bigtable-like capabilities on top of Apache Hadoop.
HBase is a tool in the Databases category of a tech stack.
HBase is an open source tool with 4K GitHub stars and 2.7K GitHub forks. Here’s a link to HBase's open source repository on GitHub

Top Alternatives to HBase

  • Cassandra

    Cassandra

    Partitioning means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster. Row store means that like relational databases, Cassandra organizes data by rows and columns. The Cassandra Query Language (CQL) is a close relative of SQL. ...

  • MongoDB

    MongoDB

    MongoDB stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema. MongoDB was also designed for high availability and scalability, with built-in replication and auto-sharding. ...

  • Hadoop

    Hadoop

    The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. ...

  • Druid

    Druid

    Druid is a distributed, column-oriented, real-time analytics data store that is commonly used to power exploratory dashboards in multi-tenant environments. Druid excels as a data warehousing solution for fast aggregate queries on petabyte sized data sets. Druid supports a variety of flexible filters, exact calculations, approximate algorithms, and other useful calculations. ...

  • Couchbase

    Couchbase

    Developed as an alternative to traditionally inflexible SQL databases, the Couchbase NoSQL database is built on an open source foundation and architected to help developers solve real-world problems and meet high scalability demands. ...

  • Apache Hive

    Apache Hive

    Hive facilitates reading, writing, and managing large datasets residing in distributed storage using SQL. Structure can be projected onto data already in storage. ...

  • RocksDB

    RocksDB

    RocksDB is an embeddable persistent key-value store for fast storage. RocksDB can also be the foundation for a client-server database but our current focus is on embedded workloads. RocksDB builds on LevelDB to be scalable to run on servers with many CPU cores, to efficiently use fast storage, to support IO-bound, in-memory and write-once workloads, and to be flexible to allow for innovation. ...

  • Redis

    Redis

    Redis is an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. ...

HBase alternatives & related posts

Cassandra logo

Cassandra

3.1K
3K
463
A partitioned row store. Rows are organized into tables with a required primary key.
3.1K
3K
+ 1
463
PROS OF CASSANDRA
  • 107
    Distributed
  • 90
    High performance
  • 77
    High availability
  • 71
    Easy scalability
  • 50
    Replication
  • 25
    Reliable
  • 24
    Multi datacenter deployments
  • 6
    Schema optional
  • 6
    OLTP
  • 5
    Open source
  • 2
    Workload separation (via MDC)
CONS OF CASSANDRA
  • 1
    Reliability of replication
  • 1
    Updates

related Cassandra posts

Thierry Schellenbach
Shared insights
on
Redis
Cassandra
RocksDB
at

1.0 of Stream leveraged Cassandra for storing the feed. Cassandra is a common choice for building feeds. Instagram, for instance started, out with Redis but eventually switched to Cassandra to handle their rapid usage growth. Cassandra can handle write heavy workloads very efficiently.

Cassandra is a great tool that allows you to scale write capacity simply by adding more nodes, though it is also very complex. This complexity made it hard to diagnose performance fluctuations. Even though we had years of experience with running Cassandra, it still felt like a bit of a black box. When building Stream 2.0 we decided to go for a different approach and build Keevo. Keevo is our in-house key-value store built upon RocksDB, gRPC and Raft.

RocksDB is a highly performant embeddable database library developed and maintained by Facebook’s data engineering team. RocksDB started as a fork of Google’s LevelDB that introduced several performance improvements for SSD. Nowadays RocksDB is a project on its own and is under active development. It is written in C++ and it’s fast. Have a look at how this benchmark handles 7 million QPS. In terms of technology it’s much more simple than Cassandra.

This translates into reduced maintenance overhead, improved performance and, most importantly, more consistent performance. It’s interesting to note that LinkedIn also uses RocksDB for their feed.

#InMemoryDatabases #DataStores #Databases

See more
Umair Iftikhar
Technical Architect at Vappar · | 3 upvotes · 50.9K views

Developing a solution that collects Telemetry Data from different devices, nearly 1000 devices minimum and maximum 12000. Each device is sending 2 packets in 1 second. This is time-series data, and this data definition and different reports are saved on PostgreSQL. Like Building information, maintenance records, etc. I want to know about the best solution. This data is required for Math and ML to run different algorithms. Also, data is raw without definitions and information stored in PostgreSQL. Initially, I went with TimescaleDB due to PostgreSQL support, but to increase in sites, I started facing many issues with timescale DB in terms of flexibility of storing data.

My major requirement is also the replication of the database for reporting and different purposes. You may also suggest other options other than Druid and Cassandra. But an open source solution is appreciated.

See more
MongoDB logo

MongoDB

55.6K
45.3K
4K
The database for giant ideas
55.6K
45.3K
+ 1
4K
PROS OF MONGODB
  • 822
    Document-oriented storage
  • 588
    No sql
  • 544
    Ease of use
  • 462
    Fast
  • 404
    High performance
  • 252
    Free
  • 213
    Open source
  • 177
    Flexible
  • 139
    Replication & high availability
  • 107
    Easy to maintain
  • 39
    Querying
  • 35
    Easy scalability
  • 34
    Auto-sharding
  • 33
    High availability
  • 29
    Map/reduce
  • 26
    Document database
  • 24
    Full index support
  • 24
    Easy setup
  • 15
    Reliable
  • 14
    Fast in-place updates
  • 13
    Agile programming, flexible, fast
  • 11
    No database migrations
  • 7
    Easy integration with Node.Js
  • 7
    Enterprise
  • 5
    Enterprise Support
  • 4
    Great NoSQL DB
  • 3
    Aggregation Framework
  • 3
    Support for many languages through different drivers
  • 3
    Drivers support is good
  • 2
    Schemaless
  • 2
    Fast
  • 2
    Awesome
  • 2
    Managed service
  • 2
    Easy to Scale
  • 1
    Consistent
CONS OF MONGODB
  • 5
    Very slowly for connected models that require joins
  • 3
    Not acid compliant
  • 1
    Proprietary query language

related MongoDB posts

Jeyabalaji Subramanian

Recently we were looking at a few robust and cost-effective ways of replicating the data that resides in our production MongoDB to a PostgreSQL database for data warehousing and business intelligence.

We set ourselves the following criteria for the optimal tool that would do this job: - The data replication must be near real-time, yet it should NOT impact the production database - The data replication must be horizontally scalable (based on the load), asynchronous & crash-resilient

Based on the above criteria, we selected the following tools to perform the end to end data replication:

We chose MongoDB Stitch for picking up the changes in the source database. It is the serverless platform from MongoDB. One of the services offered by MongoDB Stitch is Stitch Triggers. Using stitch triggers, you can execute a serverless function (in Node.js) in real time in response to changes in the database. When there are a lot of database changes, Stitch automatically "feeds forward" these changes through an asynchronous queue.

We chose Amazon SQS as the pipe / message backbone for communicating the changes from MongoDB to our own replication service. Interestingly enough, MongoDB stitch offers integration with AWS services.

In the Node.js function, we wrote minimal functionality to communicate the database changes (insert / update / delete / replace) to Amazon SQS.

Next we wrote a minimal micro-service in Python to listen to the message events on SQS, pickup the data payload & mirror the DB changes on to the target Data warehouse. We implemented source data to target data translation by modelling target table structures through SQLAlchemy . We deployed this micro-service as AWS Lambda with Zappa. With Zappa, deploying your services as event-driven & horizontally scalable Lambda service is dumb-easy.

In the end, we got to implement a highly scalable near realtime Change Data Replication service that "works" and deployed to production in a matter of few days!

See more
Robert Zuber

We use MongoDB as our primary #datastore. Mongo's approach to replica sets enables some fantastic patterns for operations like maintenance, backups, and #ETL.

As we pull #microservices from our #monolith, we are taking the opportunity to build them with their own datastores using PostgreSQL. We also use Redis to cache data we’d never store permanently, and to rate-limit our requests to partners’ APIs (like GitHub).

When we’re dealing with large blobs of immutable data (logs, artifacts, and test results), we store them in Amazon S3. We handle any side-effects of S3’s eventual consistency model within our own code. This ensures that we deal with user requests correctly while writes are in process.

See more
Hadoop logo

Hadoop

1.9K
1.9K
55
Open-source software for reliable, scalable, distributed computing
1.9K
1.9K
+ 1
55
PROS OF HADOOP
  • 38
    Great ecosystem
  • 11
    One stack to rule them all
  • 4
    Great load balancer
  • 1
    Amazon aws
  • 1
    Java syntax
CONS OF HADOOP
    Be the first to leave a con

    related Hadoop posts

    Conor Myhrvold
    Tech Brand Mgr, Office of CTO at Uber · | 7 upvotes · 919.1K views

    Why we built Marmaray, an open source generic data ingestion and dispersal framework and library for Apache Hadoop :

    Built and designed by our Hadoop Platform team, Marmaray is a plug-in-based framework built on top of the Hadoop ecosystem. Users can add support to ingest data from any source and disperse to any sink leveraging the use of Apache Spark . The name, Marmaray, comes from a tunnel in Turkey connecting Europe and Asia. Similarly, we envisioned Marmaray within Uber as a pipeline connecting data from any source to any sink depending on customer preference:

    https://eng.uber.com/marmaray-hadoop-ingestion-open-source/

    (Direct GitHub repo: https://github.com/uber/marmaray Kafka Kafka Manager )

    See more
    Shared insights
    on
    Kafka
    Hadoop
    at

    The early data ingestion pipeline at Pinterest used Kafka as the central message transporter, with the app servers writing messages directly to Kafka, which then uploaded log files to S3.

    For databases, a custom Hadoop streamer pulled database data and wrote it to S3.

    Challenges cited for this infrastructure included high operational overhead, as well as potential data loss occurring when Kafka broker outages led to an overflow of in-memory message buffering.

    See more
    Druid logo

    Druid

    282
    596
    27
    Fast column-oriented distributed data store
    282
    596
    + 1
    27
    PROS OF DRUID
    • 13
      Real Time Aggregations
    • 4
      OLAP
    • 4
      Batch and Real-Time Ingestion
    • 3
      OLAP + OLTP
    • 2
      Combining stream and historical analytics
    • 1
      OLTP
    CONS OF DRUID
    • 2
      Limited sql support
    • 1
      Complexity
    • 1
      Joins are not supported well

    related Druid posts

    Umair Iftikhar
    Technical Architect at Vappar · | 3 upvotes · 50.9K views

    Developing a solution that collects Telemetry Data from different devices, nearly 1000 devices minimum and maximum 12000. Each device is sending 2 packets in 1 second. This is time-series data, and this data definition and different reports are saved on PostgreSQL. Like Building information, maintenance records, etc. I want to know about the best solution. This data is required for Math and ML to run different algorithms. Also, data is raw without definitions and information stored in PostgreSQL. Initially, I went with TimescaleDB due to PostgreSQL support, but to increase in sites, I started facing many issues with timescale DB in terms of flexibility of storing data.

    My major requirement is also the replication of the database for reporting and different purposes. You may also suggest other options other than Druid and Cassandra. But an open source solution is appreciated.

    See more
    Couchbase logo

    Couchbase

    352
    461
    101
    Document-Oriented NoSQL Database
    352
    461
    + 1
    101
    PROS OF COUCHBASE
    • 18
      High performance
    • 17
      Flexible data model, easy scalability, extremely fast
    • 8
      Mobile app support
    • 6
      You can query it with Ansi-92 SQL
    • 5
      All nodes can be read/write
    • 4
      Open source, community and enterprise editions
    • 4
      Local cache capability
    • 4
      Equal nodes in cluster, allowing fast, flexible changes
    • 4
      Both a key-value store and document (JSON) db
    • 3
      Automatic configuration of sharding
    • 3
      SDKs in popular programming languages
    • 3
      Elasticsearch connector
    • 3
      Easy setup
    • 3
      Web based management, query and monitoring panel
    • 3
      Linearly scalable, useful to large number of tps
    • 3
      Easy cluster administration
    • 3
      Cross data center replication
    • 2
      NoSQL
    • 2
      DBaaS available
    • 2
      Map reduce views
    • 1
      FTS + SQL together
    CONS OF COUCHBASE
    • 3
      Terrible query language

    related Couchbase posts

    Gabriel Pa

    We implemented our first large scale EPR application from naologic.com using CouchDB .

    Very fast, replication works great, doesn't consume much RAM, queries are blazing fast but we found a problem: the queries were very hard to write, it took a long time to figure out the API, we had to go and write our own @nodejs library to make it work properly.

    It lost most of its support. Since then, we migrated to Couchbase and the learning curve was steep but all worth it. Memcached indexing out of the box, full text search works great.

    See more
    Ilias Mentzelos
    Software Engineer at Plum Fintech · | 8 upvotes · 10K views
    Shared insights
    on
    MongoDB
    Couchbase

    Hey, we want to build a referral campaign mechanism that will probably contain millions of records within the next few years. We want fast read access based on IDs or some indexes, and isolation is crucial as some listeners will try to update the same document at the same time. What's your suggestion between Couchbase and MongoDB? Thanks!

    See more
    Apache Hive logo

    Apache Hive

    322
    309
    0
    Data Warehouse Software for Reading, Writing, and Managing Large Datasets
    322
    309
    + 1
    0
    PROS OF APACHE HIVE
      Be the first to leave a pro
      CONS OF APACHE HIVE
        Be the first to leave a con

        related Apache Hive posts

        Ashish Singh
        Tech Lead, Big Data Platform at Pinterest · | 35 upvotes · 725.2K views

        To provide employees with the critical need of interactive querying, we’ve worked with Presto, an open-source distributed SQL query engine, over the years. Operating Presto at Pinterest’s scale has involved resolving quite a few challenges like, supporting deeply nested and huge thrift schemas, slow/ bad worker detection and remediation, auto-scaling cluster, graceful cluster shutdown and impersonation support for ldap authenticator.

        Our infrastructure is built on top of Amazon EC2 and we leverage Amazon S3 for storing our data. This separates compute and storage layers, and allows multiple compute clusters to share the S3 data.

        We have hundreds of petabytes of data and tens of thousands of Apache Hive tables. Our Presto clusters are comprised of a fleet of 450 r4.8xl EC2 instances. Presto clusters together have over 100 TBs of memory and 14K vcpu cores. Within Pinterest, we have close to more than 1,000 monthly active users (out of total 1,600+ Pinterest employees) using Presto, who run about 400K queries on these clusters per month.

        Each query submitted to Presto cluster is logged to a Kafka topic via Singer. Singer is a logging agent built at Pinterest and we talked about it in a previous post. Each query is logged when it is submitted and when it finishes. When a Presto cluster crashes, we will have query submitted events without corresponding query finished events. These events enable us to capture the effect of cluster crashes over time.

        Each Presto cluster at Pinterest has workers on a mix of dedicated AWS EC2 instances and Kubernetes pods. Kubernetes platform provides us with the capability to add and remove workers from a Presto cluster very quickly. The best-case latency on bringing up a new worker on Kubernetes is less than a minute. However, when the Kubernetes cluster itself is out of resources and needs to scale up, it can take up to ten minutes. Some other advantages of deploying on Kubernetes platform is that our Presto deployment becomes agnostic of cloud vendor, instance types, OS, etc.

        #BigData #AWS #DataScience #DataEngineering

        See more
        RocksDB logo

        RocksDB

        78
        173
        10
        Embeddable persistent key-value store for fast storage, developed and maintained by Facebook Database Engineering Team
        78
        173
        + 1
        10
        PROS OF ROCKSDB
        • 4
          Very fast
        • 3
          Made by Facebook
        • 2
          Consistent performance
        • 1
          Ability to add logic to the database layer where needed
        CONS OF ROCKSDB
          Be the first to leave a con

          related RocksDB posts

          Thierry Schellenbach
          Shared insights
          on
          Redis
          Cassandra
          RocksDB
          at

          1.0 of Stream leveraged Cassandra for storing the feed. Cassandra is a common choice for building feeds. Instagram, for instance started, out with Redis but eventually switched to Cassandra to handle their rapid usage growth. Cassandra can handle write heavy workloads very efficiently.

          Cassandra is a great tool that allows you to scale write capacity simply by adding more nodes, though it is also very complex. This complexity made it hard to diagnose performance fluctuations. Even though we had years of experience with running Cassandra, it still felt like a bit of a black box. When building Stream 2.0 we decided to go for a different approach and build Keevo. Keevo is our in-house key-value store built upon RocksDB, gRPC and Raft.

          RocksDB is a highly performant embeddable database library developed and maintained by Facebook’s data engineering team. RocksDB started as a fork of Google’s LevelDB that introduced several performance improvements for SSD. Nowadays RocksDB is a project on its own and is under active development. It is written in C++ and it’s fast. Have a look at how this benchmark handles 7 million QPS. In terms of technology it’s much more simple than Cassandra.

          This translates into reduced maintenance overhead, improved performance and, most importantly, more consistent performance. It’s interesting to note that LinkedIn also uses RocksDB for their feed.

          #InMemoryDatabases #DataStores #Databases

          See more
          Redis logo

          Redis

          37.6K
          27.6K
          3.9K
          An in-memory database that persists on disk
          37.6K
          27.6K
          + 1
          3.9K
          PROS OF REDIS
          • 877
            Performance
          • 535
            Super fast
          • 511
            Ease of use
          • 442
            In-memory cache
          • 321
            Advanced key-value cache
          • 190
            Open source
          • 179
            Easy to deploy
          • 163
            Stable
          • 153
            Free
          • 120
            Fast
          • 40
            High-Performance
          • 39
            High Availability
          • 34
            Data Structures
          • 32
            Very Scalable
          • 23
            Replication
          • 20
            Great community
          • 19
            Pub/Sub
          • 17
            "NoSQL" key-value data store
          • 14
            Hashes
          • 12
            Sets
          • 10
            Sorted Sets
          • 9
            Lists
          • 8
            BSD licensed
          • 8
            NoSQL
          • 7
            Async replication
          • 7
            Integrates super easy with Sidekiq for Rails background
          • 7
            Bitmaps
          • 6
            Open Source
          • 6
            Keys with a limited time-to-live
          • 5
            Strings
          • 5
            Lua scripting
          • 4
            Awesomeness for Free!
          • 4
            Hyperloglogs
          • 3
            outstanding performance
          • 3
            Runs server side LUA
          • 3
            Networked
          • 3
            LRU eviction of keys
          • 3
            Written in ANSI C
          • 3
            Feature Rich
          • 3
            Transactions
          • 2
            Data structure server
          • 2
            Performance & ease of use
          • 1
            Existing Laravel Integration
          • 1
            Automatic failover
          • 1
            Easy to use
          • 1
            Object [key/value] size each 500 MB
          • 1
            Simple
          • 1
            Channels concept
          • 1
            Scalable
          • 1
            Temporarily kept on disk
          • 1
            Dont save data if no subscribers are found
          • 0
            Jk
          CONS OF REDIS
          • 12
            Cannot query objects directly
          • 1
            No WAL
          • 1
            No secondary indexes for non-numeric data types

          related Redis posts

          Robert Zuber

          We use MongoDB as our primary #datastore. Mongo's approach to replica sets enables some fantastic patterns for operations like maintenance, backups, and #ETL.

          As we pull #microservices from our #monolith, we are taking the opportunity to build them with their own datastores using PostgreSQL. We also use Redis to cache data we’d never store permanently, and to rate-limit our requests to partners’ APIs (like GitHub).

          When we’re dealing with large blobs of immutable data (logs, artifacts, and test results), we store them in Amazon S3. We handle any side-effects of S3’s eventual consistency model within our own code. This ensures that we deal with user requests correctly while writes are in process.

          See more

          I'm working as one of the engineering leads in RunaHR. As our platform is a Saas, we thought It'd be good to have an API (We chose Ruby and Rails for this) and a SPA (built with React and Redux ) connected. We started the SPA with Create React App since It's pretty easy to start.

          We use Jest as the testing framework and react-testing-library to test React components. In Rails we make tests using RSpec.

          Our main database is PostgreSQL, but we also use MongoDB to store some type of data. We started to use Redis  for cache and other time sensitive operations.

          We have a couple of extra projects: One is an Employee app built with React Native and the other is an internal back office dashboard built with Next.js for the client and Python in the backend side.

          Since we have different frontend apps we have found useful to have Bit to document visual components and utils in JavaScript.

          See more