Apache Storm logo

Apache Storm

Distributed and fault-tolerant realtime computation
130
116
+ 1
18

What is Apache Storm?

Apache Storm is a free and open source distributed realtime computation system. Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing. Storm has many use cases: realtime analytics, online machine learning, continuous computation, distributed RPC, ETL, and more. Storm is fast: a benchmark clocked it at over a million tuples processed per second per node. It is scalable, fault-tolerant, guarantees your data will be processed, and is easy to set up and operate.
Apache Storm is a tool in the Stream Processing category of a tech stack.
Apache Storm is an open source tool with 5.9K GitHub stars and 4K GitHub forks. Here鈥檚 a link to Apache Storm's open source repository on GitHub

Who uses Apache Storm?

Companies
55 companies reportedly use Apache Storm in their tech stacks, including Spotify, Twitter, and Yelp.

Developers
69 developers on StackShare have stated that they use Apache Storm.

Apache Storm Integrations

Why developers like Apache Storm?

Here鈥檚 a list of reasons why companies and developers use Apache Storm
Apache Storm Reviews

Here are some stack decisions, common use cases and reviews by companies and developers who chose Apache Storm in their tech stack.

Marc Bollinger
Marc Bollinger
Infra & Data Eng Manager at Lumosity | 4 upvotes 76.8K views
atLumosityLumosity
Node.js
Node.js
Ruby
Ruby
Kafka
Kafka
Scala
Scala
Apache Storm
Apache Storm
Heron
Heron
Redis
Redis
Pulsar
Pulsar

Lumosity is home to the world's largest cognitive training database, a responsibility we take seriously. For most of the company's history, our analysis of user behavior and training data has been powered by an event stream--first a simple Node.js pub/sub app, then a heavyweight Ruby app with stronger durability. Both supported decent throughput and latency, but they lacked some major features supported by existing open-source alternatives: replaying existing messages (also lacking in most message queue-based solutions), scaling out many different readers for the same stream, the ability to leverage existing solutions for reading and writing, and possibly most importantly: the ability to hire someone externally who already had expertise.

We ultimately migrated to Kafka in early- to mid-2016, citing both industry trends in companies we'd talked to with similar durability and throughput needs, the extremely strong documentation and community. We pored over Kyle Kingsbury's Jepsen post (https://aphyr.com/posts/293-jepsen-Kafka), as well as Jay Kreps' follow-up (http://blog.empathybox.com/post/62279088548/a-few-notes-on-kafka-and-jepsen), talked at length with Confluent folks and community members, and still wound up running parallel systems for quite a long time, but ultimately, we've been very, very happy. Understanding the internals and proper levers takes some commitment, but it's taken very little maintenance once configured. Since then, the Confluent Platform community has grown and grown; we've gone from doing most development using custom Scala consumers and producers to being 60/40 Kafka Streams/Connects.

We originally looked into Storm / Heron , and we'd moved on from Redis pub/sub. Heron looks great, but we already had a programming model across services that was more akin to consuming a message consumers than required a topology of bolts, etc. Heron also had just come out while we were starting to migrate things, and the community momentum and direction of Kafka felt more substantial than the older Storm. If we were to start the process over again today, we might check out Pulsar , although the ecosystem is much younger.

To find out more, read our 2017 engineering blog post about the migration!

See more
Apache Storm
Apache Storm

Real-time analytics are much better than periodically run batch jobs, so recently we open sourced Pyleus which allows anyone to write Storm topologies using Python. Storm

See more

Apache Storm's Features

  • Storm integrates with the queueing and database technologies you already use
  • Simple API
  • Scalable
  • Fault tolerant
  • Guarantees data processing
  • Use with any language
  • Easy to deploy and operate
  • Free and open source

Apache Storm Alternatives & Comparisons

What are some alternatives to Apache Storm?
Apache Spark
Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning.
Kafka
Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.
Amazon Kinesis
Amazon Kinesis can collect and process hundreds of gigabytes of data per second from hundreds of thousands of sources, allowing you to easily write applications that process information in real-time, from sources such as web site click-streams, marketing and financial information, manufacturing instrumentation and social media, and operational logs and metering data.
Apache Flume
It is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application.
Apache Flink
Apache Flink is an open source system for fast and versatile data analytics in clusters. Flink supports batch and streaming analytics, in one system. Analytical programs can be written in concise and elegant APIs in Java and Scala.
See all alternatives

Apache Storm's Followers
116 developers follow Apache Storm to keep up with related blogs and decisions.
Nurullah 脰zdemir
mlbn_dist
Diane Napolitano
John Alton
Mohamma76685757
脡ric Le Merdy
Ashish Tanwer
Gabriel Georgescu
Ashutos36306861
Matthew Stokeley