Need advice about which tool to choose?Ask the StackShare community!
Hadoop vs Redis: What are the differences?
What is Hadoop? Open-source software for reliable, scalable, distributed computing. The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.
What is Redis? An in-memory database that persists on disk. Redis is an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.
Hadoop and Redis are primarily classified as "Databases" and "In-Memory Databases" tools respectively.
"Great ecosystem" is the primary reason why developers consider Hadoop over the competitors, whereas "Performance" was stated as the key factor in picking Redis.
Hadoop and Redis are both open source tools. It seems that Redis with 37.1K GitHub stars and 14.3K forks on GitHub has more adoption than Hadoop with 9.18K GitHub stars and 5.74K GitHub forks.
reddit, Instacart, and Slack are some of the popular companies that use Redis, whereas Hadoop is used by Slack, Shopify, and SendGrid. Redis has a broader approval, being mentioned in 3239 company stacks & 1732 developers stacks; compared to Hadoop, which is listed in 237 company stacks and 116 developer stacks.
What is Hadoop?
What is Redis?
Need advice about which tool to choose?Ask the StackShare community!
Why do developers choose Hadoop?
Why do developers choose Redis?
- Performance845
- Super fast520
- Ease of use502
- Open source186
Sign up to add, upvote and see more prosMake informed product decisions
What are the cons of using Hadoop?
What are the cons of using Redis?
Sign up to get full access to all the companiesMake informed product decisions
What tools integrate with Hadoop?
What tools integrate with Redis?
Sign up to get full access to all the tool integrationsMake informed product decisions
RedisRedis is a good caching tool for a cluster, but our application had performance issues while using Aws Elasticache Redis since some page had 3000 cache hits per a page load and Redis just couldn't quickly process them all in once + latency and object deseialization time - page load took 8-9 seconds. We create a custom hybrid caching based on Redis and EhCache which worked great for our goals. Check it out on github, it's called HybriCache - https://github.com/batir-akhmerov/hybricache.
RedisRedis is used for storing all ephemeral (that's data you don't necessarily want to store permanently) user data, such as mapping of session IDs (stored in cookies) to current session variables at Cloudcraft.co. The many datastructures supported by Redis also makes it an excellent caching and realtime statistics layer. It doesn't hurt that the author, Antirez, is the nicest guy ever! These days, I would be really hard pressed to find any situation where I would pick something like Memcached over Redis.
RedisTrello uses Redis for ephemeral data that needs to be shared between server processes but not persisted to disk. Things like the activity level of a session or a temporary OpenID key are stored in Redis, and the application is built to recover gracefully if any of these (or all of them) are lost. We run with allkeys-lru enabled and about five times as much space as its actual working set needs, so Redis automatically discards data that hasn’t been accessed lately, and reconstructs it when necessary.
HadoopThe MapReduce workflow starts to process experiment data nightly when data of the previous day is copied over from Kafka. At this time, all the raw log requests are transformed into meaningful experiment results and in-depth analysis. To populate experiment data for the dashboard, we have around 50 jobs running to do all the calculations and transforms of data.
RedisThe UI has message inbox that is sent a message when you get a new badge, receive a message, significant event, etc. Done using WebSockets and is powered by redis. Redis has 2 slaves, SQL has 2 replicas, tag engine has 3 nodes, elastic has 3 nodes - any other service has high availability as well (and exists in both data centers).
RedisRedis makes certain operations very easy. When I need a high-availability store, I typically look elsewhere, but for rapid development with the ability to land on your feet in prod, Redis is great. The available data types make it easy to build non-trivial indexes that would require complex queries in postgres.
RedisI use Redis for cacheing, data storage, mining and augmentation, proprietary distributed event system for disparate apps and services to talk to each other, and more. Redis has some very useful native data types for tracking, slicing and dicing information.
Hadoopin 2009 we open sourced mrjob, which allows any engineer to write a MapReduce job without contending for resources. We’re only limited by the amount of machines in an Amazon data center (which is an issue we’ve rarely encountered).
HadoopThe massive volume of discovery data that powers Pinterest and enables people to save Pins, create boards and follow other users, is generated through daily Hadoop jobs...
HadoopImporting/Exporting data, interpreting results. Possible integration with SAS
HadoopTBD. Good to have I think. Analytics on loads of data, recommendations?