Hadoop vs ZeroMQ

Overview

Hadoop

Stacks2.7K

Followers2.3K

Votes56

GitHub Stars15.3K

Forks9.1K

ZeroMQ

Stacks258

Followers586

Votes71

GitHub Stars10.6K

Forks2.5K

Hadoop vs ZeroMQ: What are the differences?

Hadoop: Open-source software for reliable, scalable, distributed computing. The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage; ZeroMQ: Fast, lightweight messaging library that allows you to design complex communication system without much effort. The 0MQ lightweight messaging kernel is a library which extends the standard socket interfaces with features traditionally provided by specialised messaging middleware products. 0MQ sockets provide an abstraction of asynchronous message queues, multiple messaging patterns, message filtering (subscriptions), seamless access to multiple transport protocols and more.

Hadoop and ZeroMQ are primarily classified as "Databases" and "Message Queue" tools respectively.

"Great ecosystem" is the primary reason why developers consider Hadoop over the competitors, whereas "Fast" was stated as the key factor in picking ZeroMQ.

Hadoop and ZeroMQ are both open source tools. Hadoop with 9.27K GitHub stars and 5.78K forks on GitHub appears to be more popular than ZeroMQ with 5.34K GitHub stars and 1.57K GitHub forks.

According to the StackShare community, Hadoop has a broader approval, being mentioned in 237 company stacks & 127 developers stacks; compared to ZeroMQ, which is listed in 35 company stacks and 12 developer stacks.

Share your Stack

Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.

View Docs

CLI (Node.js)

Manual

Advice on Hadoop, ZeroMQ

pionell

Sep 16, 2020

Needs adviceon

MariaDB

I have a lot of data that's currently sitting in a MariaDB database, a lot of tables that weigh 200gb with indexes. Most of the large tables have a date column which is always filtered, but there are usually 4-6 additional columns that are filtered and used for statistics. I'm trying to figure out the best tool for storing and analyzing large amounts of data. Preferably self-hosted or a cheap solution. The current problem I'm running into is speed. Even with pretty good indexes, if I'm trying to load a large dataset, it's pretty slow.

159k views159k

Comments

Detailed Comparison

Hadoop	ZeroMQ
The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.	The 0MQ lightweight messaging kernel is a library which extends the standard socket interfaces with features traditionally provided by specialised messaging middleware products. 0MQ sockets provide an abstraction of asynchronous message queues, multiple messaging patterns, message filtering (subscriptions), seamless access to multiple transport protocols and more.
-	Connect your code in any language, on any platform.;Carries messages across inproc, IPC, TCP, TPIC, multicast.;Smart patterns like pub-sub, push-pull, and router-dealer.;High-speed asynchronous I/O engines, in a tiny library.;Backed by a large and active open source community.;Supports every modern language and platform.;Build any architecture: centralized, distributed, small, or large.;Free software with full commercial support.
Statistics
GitHub Stars 15.3K	GitHub Stars 10.6K
GitHub Forks 9.1K	GitHub Forks 2.5K
Stacks 2.7K	Stacks 258
Followers 2.3K	Followers 586
Votes 56	Votes 71
Pros & Cons
Pros 39 Great ecosystem 11 One stack to rule them all 4 Great load balancer 1 Java syntax 1 Amazon aws	Pros 23 Fast 20 Lightweight 11 Transport agnostic 7 No broker required 4 Low latency Cons 5 No message durability 3 Not a very reliable system - message delivery wise 1 M x N problem with M producers and N consumers

What are some alternatives to Hadoop, ZeroMQ?

MongoDB

MongoDB stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema. MongoDB was also designed for high availability and scalability, with built-in replication and auto-sharding.

MySQL

The MySQL software delivers a very fast, multi-threaded, multi-user, and robust SQL (Structured Query Language) database server. MySQL Server is intended for mission-critical, heavy-load production systems as well as for embedding into mass-deployed software.

PostgreSQL

PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions.

Kafka

Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.

RabbitMQ

RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received.

Microsoft SQL Server

Microsoft® SQL Server is a database management and analysis system for e-commerce, line-of-business, and data warehousing solutions.

SQLite

SQLite is an embedded SQL database engine. Unlike most other SQL databases, SQLite does not have a separate server process. SQLite reads and writes directly to ordinary disk files. A complete SQL database with multiple tables, indices, triggers, and views, is contained in a single disk file.

Cassandra

Partitioning means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster. Row store means that like relational databases, Cassandra organizes data by rows and columns. The Cassandra Query Language (CQL) is a close relative of SQL.

Memcached

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

MariaDB

Started by core members of the original MySQL team, MariaDB actively works with outside developers to deliver the most featureful, stable, and sanely licensed open SQL server in the industry. MariaDB is designed as a drop-in replacement of MySQL(R) with more features, new storage engines, fewer bugs, and better performance.

Related Comparisons

"Great ecosystem" is the primary reason why developers consider Hadoop over the competitors, whereas "Fast" was stated as the key factor in picking ZeroMQ.

Hadoop vs ZeroMQ