Citus vs Hadoop

Get Advice Icon

Need advice about which tool to choose?Ask the StackShare community!

Citus
Citus

28
26
+ 1
8
Hadoop
Hadoop

1.1K
887
+ 1
48
Add tool

Citus vs Hadoop: What are the differences?

Citus: Worry-free Postgres for SaaS. Built to scale out. Citus is worry-free Postgres for SaaS. Made to scale out, Citus is an extension to Postgres that distributes queries across any number of servers. Citus is available as open source, as on-prem software, and as a fully-managed service; Hadoop: Open-source software for reliable, scalable, distributed computing. The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.

Citus and Hadoop belong to "Databases" category of the tech stack.

"Multi-core Parallel Processing" is the primary reason why developers consider Citus over the competitors, whereas "Great ecosystem" was stated as the key factor in picking Hadoop.

Citus and Hadoop are both open source tools. Hadoop with 9.26K GitHub stars and 5.78K forks on GitHub appears to be more popular than Citus with 3.64K GitHub stars and 273 GitHub forks.

What is Citus?

It's an extension to Postgres that distributes data and queries in a cluster of multiple machines. Its query engine parallelizes incoming SQL queries across these servers to enable human real-time (less than a second) responses on large datasets.

What is Hadoop?

The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.
Get Advice Icon

Need advice about which tool to choose?Ask the StackShare community!

Why do developers choose Citus?
Why do developers choose Hadoop?
    Be the first to leave a con
      Be the first to leave a con
      What companies use Citus?
      What companies use Hadoop?

      Sign up to get full access to all the companiesMake informed product decisions

      What tools integrate with Citus?
      What tools integrate with Hadoop?

      Sign up to get full access to all the tool integrationsMake informed product decisions

      What are some alternatives to Citus and Hadoop?
      TimescaleDB
      TimescaleDB: An open-source database built for analyzing time-series data with the power and convenience of SQL — on premise, at the edge, or in the cloud.
      CockroachDB
      Cockroach Labs is the company building CockroachDB, an open source, survivable, strongly consistent, scale-out SQL database.
      MySQL
      The MySQL software delivers a very fast, multi-threaded, multi-user, and robust SQL (Structured Query Language) database server. MySQL Server is intended for mission-critical, heavy-load production systems as well as for embedding into mass-deployed software.
      PostgreSQL
      PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions.
      MongoDB
      MongoDB stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema. MongoDB was also designed for high availability and scalability, with built-in replication and auto-sharding.
      See all alternatives
      Decisions about Citus and Hadoop
      Dan Robinson
      Dan Robinson
      at Heap, Inc. · | 16 upvotes · 53.6K views
      atHeapHeap
      Citus
      Citus
      PostgreSQL
      PostgreSQL
      #DataStores
      #Databases

      PostgreSQL was an easy early decision for the founding team. The relational data model fit the types of analyses they would be doing: filtering, grouping, joining, etc., and it was the database they knew best.

      Shortly after adopting PG, they discovered Citus, which is a tool that makes it easy to distribute queries. Although it was a young project and a fork of Postgres at that point, Dan says the team was very available, highly expert, and it wouldn’t be very difficult to move back to PG if they needed to.

      The stuff they forked was in query execution. You could treat the worker nodes like regular PG instances. Citus also gave them a ton of flexibility to make queries fast, and again, they felt the data model was the best fit for their application.

      #DataStores #Databases

      See more
      Dan Robinson
      Dan Robinson
      at Heap, Inc. · | 14 upvotes · 46.6K views
      atHeapHeap
      Heap
      Heap
      Node.js
      Node.js
      Kafka
      Kafka
      PostgreSQL
      PostgreSQL
      Citus
      Citus
      #MessageQueue
      #Databases
      #FrameworksFullStack

      At Heap, we searched for an existing tool that would allow us to express the full range of analyses we needed, index the event definitions that made up the analyses, and was a mature, natively distributed system.

      After coming up empty on this search, we decided to compromise on the “maturity” requirement and build our own distributed system around Citus and sharded PostgreSQL. It was at this point that we also introduced Kafka as a queueing layer between the Node.js application servers and Postgres.

      If we could go back in time, we probably would have started using Kafka on day one. One of the biggest benefits in adopting Kafka has been the peace of mind that it brings. In an analytics infrastructure, it’s often possible to make data ingestion idempotent.

      In Heap’s case, that means that, if anything downstream from Kafka goes down, we won’t lose any data – it’s just going to take a bit longer to get to its destination. We also learned that you want the path between data hitting your servers and your initial persistence layer (in this case, Kafka) to be as short and simple as possible, since that is the surface area where a failure means you can lose customer data. We learned that it’s a very good fit for an analytics tool, since you can handle a huge number of incoming writes with relatively low latency. Kafka also gives you the ability to “replay” the data flow: it’s like a commit log for your whole infrastructure.

      #MessageQueue #Databases #FrameworksFullStack

      See more
      StackShare Editors
      StackShare Editors
      Apache Thrift
      Apache Thrift
      Kotlin
      Kotlin
      Presto
      Presto
      HHVM (HipHop Virtual Machine)
      HHVM (HipHop Virtual Machine)
      gRPC
      gRPC
      Kubernetes
      Kubernetes
      Apache Spark
      Apache Spark
      Airflow
      Airflow
      Terraform
      Terraform
      Hadoop
      Hadoop
      Swift
      Swift
      Hack
      Hack
      Memcached
      Memcached
      Consul
      Consul
      Chef
      Chef
      Prometheus
      Prometheus

      Since the beginning, Cal Henderson has been the CTO of Slack. Earlier this year, he commented on a Quora question summarizing their current stack.

      Apps
      • Web: a mix of JavaScript/ES6 and React.
      • Desktop: And Electron to ship it as a desktop application.
      • Android: a mix of Java and Kotlin.
      • iOS: written in a mix of Objective C and Swift.
      Backend
      • The core application and the API written in PHP/Hack that runs on HHVM.
      • The data is stored in MySQL using Vitess.
      • Caching is done using Memcached and MCRouter.
      • The search service takes help from SolrCloud, with various Java services.
      • The messaging system uses WebSockets with many services in Java and Go.
      • Load balancing is done using HAproxy with Consul for configuration.
      • Most services talk to each other over gRPC,
      • Some Thrift and JSON-over-HTTP
      • Voice and video calling service was built in Elixir.
      Data warehouse
      • Built using open source tools including Presto, Spark, Airflow, Hadoop and Kafka.
      Etc
      See more
      Interest over time
      Reviews of Citus and Hadoop
      No reviews found
      How developers use Citus and Hadoop
      Avatar of Pinterest
      Pinterest uses HadoopHadoop

      The MapReduce workflow starts to process experiment data nightly when data of the previous day is copied over from Kafka. At this time, all the raw log requests are transformed into meaningful experiment results and in-depth analysis. To populate experiment data for the dashboard, we have around 50 jobs running to do all the calculations and transforms of data.

      Avatar of Yelp
      Yelp uses HadoopHadoop

      in 2009 we open sourced mrjob, which allows any engineer to write a MapReduce job without contending for resources. We’re only limited by the amount of machines in an Amazon data center (which is an issue we’ve rarely encountered).

      Avatar of Pinterest
      Pinterest uses HadoopHadoop

      The massive volume of discovery data that powers Pinterest and enables people to save Pins, create boards and follow other users, is generated through daily Hadoop jobs...

      Avatar of Robert Brown
      Robert Brown uses HadoopHadoop

      Importing/Exporting data, interpreting results. Possible integration with SAS

      Avatar of Rohith Nandakumar
      Rohith Nandakumar uses HadoopHadoop

      TBD. Good to have I think. Analytics on loads of data, recommendations?

      How much does Citus cost?
      How much does Hadoop cost?
      Pricing unavailable