Apache Parquet vs Hadoop

Overview

Hadoop

Stacks2.7K

Followers2.3K

Votes56

GitHub Stars15.3K

Forks9.1K

Apache Parquet

Stacks97

Followers190

Votes0

Hadoop vs Apache Parquet: What are the differences?

Hadoop: Open-source software for reliable, scalable, distributed computing. The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage; Apache Parquet: *A free and open-source column-oriented data storage format *. It is a columnar storage format available to any project in the Hadoop ecosystem, regardless of the choice of data processing framework, data model or programming language.

Hadoop can be classified as a tool in the "Databases" category, while Apache Parquet is grouped under "Big Data Tools".

Hadoop and Apache Parquet are both open source tools. It seems that Hadoop with 9.5K GitHub stars and 5.92K forks on GitHub has more adoption than Apache Parquet with 918 GitHub stars and 805 GitHub forks.

Airbnb, Uber Technologies, and Netflix are some of the popular companies that use Hadoop, whereas Apache Parquet is used by Grandata, Yotpo, and Bigabid. Hadoop has a broader approval, being mentioned in 313 company stacks & 658 developers stacks; compared to Apache Parquet, which is listed in 6 company stacks and 7 developer stacks.

Share your Stack

Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.

View Docs

CLI (Node.js)

Manual

Detailed Comparison

Hadoop	Apache Parquet
The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.	It is a columnar storage format available to any project in the Hadoop ecosystem, regardless of the choice of data processing framework, data model or programming language.
-	Columnar storage format;Type-specific encoding; Pig integration; Cascading integration; Crunch integration; Apache Arrow integration; Apache Scrooge integration;Adaptive dictionary encoding; Predicate pushdown; Column stats
Statistics
GitHub Stars 15.3K	GitHub Stars -
GitHub Forks 9.1K	GitHub Forks -
Stacks 2.7K	Stacks 97
Followers 2.3K	Followers 190
Votes 56	Votes 0
Pros & Cons
Pros 39 Great ecosystem 11 One stack to rule them all 4 Great load balancer 1 Java syntax 1 Amazon aws	No community feedback yet
Integrations
No integrations available	Java Apache Impala Apache Thrift Apache Hive Pig

What are some alternatives to Hadoop, Apache Parquet?

MongoDB

MongoDB stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema. MongoDB was also designed for high availability and scalability, with built-in replication and auto-sharding.

MySQL

The MySQL software delivers a very fast, multi-threaded, multi-user, and robust SQL (Structured Query Language) database server. MySQL Server is intended for mission-critical, heavy-load production systems as well as for embedding into mass-deployed software.

PostgreSQL

PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions.

Microsoft SQL Server

Microsoft® SQL Server is a database management and analysis system for e-commerce, line-of-business, and data warehousing solutions.

SQLite

SQLite is an embedded SQL database engine. Unlike most other SQL databases, SQLite does not have a separate server process. SQLite reads and writes directly to ordinary disk files. A complete SQL database with multiple tables, indices, triggers, and views, is contained in a single disk file.

Cassandra

Partitioning means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster. Row store means that like relational databases, Cassandra organizes data by rows and columns. The Cassandra Query Language (CQL) is a close relative of SQL.

Memcached

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

MariaDB

Started by core members of the original MySQL team, MariaDB actively works with outside developers to deliver the most featureful, stable, and sanely licensed open SQL server in the industry. MariaDB is designed as a drop-in replacement of MySQL(R) with more features, new storage engines, fewer bugs, and better performance.

RethinkDB

RethinkDB is built to store JSON documents, and scale to multiple machines with very little effort. It has a pleasant query language that supports really useful queries like table joins and group by, and is easy to setup and learn.

ArangoDB

A distributed free and open-source database with a flexible data model for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.