Get Advice Icon

Need advice about which tool to choose?Ask the StackShare community!

Airflow
Airflow

385
356
+ 1
20
Apache Beam
Apache Beam

25
23
+ 1
0
Add tool

Airflow vs Apache Beam: What are the differences?

Airflow: A platform to programmaticaly author, schedule and monitor data pipelines, by Airbnb. Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command lines utilities makes performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress and troubleshoot issues when needed; Apache Beam: A unified programming model. It implements batch and streaming data processing jobs that run on any execution engine. It executes pipelines on multiple execution environments.

Airflow and Apache Beam can be primarily classified as "Workflow Manager" tools.

Airflow is an open source tool with 13.3K GitHub stars and 4.91K GitHub forks. Here's a link to Airflow's open source repository on GitHub.

According to the StackShare community, Airflow has a broader approval, being mentioned in 98 company stacks & 162 developers stacks; compared to Apache Beam, which is listed in 9 company stacks and 4 developer stacks.

- No public GitHub repository available -

What is Airflow?

Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command lines utilities makes performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress and troubleshoot issues when needed.

What is Apache Beam?

It implements batch and streaming data processing jobs that run on any execution engine. It executes pipelines on multiple execution environments.
Get Advice Icon

Need advice about which tool to choose?Ask the StackShare community!

Why do developers choose Airflow?
Why do developers choose Apache Beam?
    Be the first to leave a pro

    Sign up to add, upvote and see more prosMake informed product decisions

      Be the first to leave a con
        Be the first to leave a con
        What companies use Airflow?
        What companies use Apache Beam?

        Sign up to get full access to all the companiesMake informed product decisions

        What tools integrate with Airflow?
        What tools integrate with Apache Beam?
          No integrations found
          What are some alternatives to Airflow and Apache Beam?
          Luigi
          It is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
          Apache NiFi
          An easy to use, powerful, and reliable system to process and distribute data. It supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic.
          Jenkins
          In a nutshell Jenkins CI is the leading open-source continuous integration server. Built with Java, it provides over 300 plugins to support building and testing virtually any project.
          AWS Step Functions
          AWS Step Functions makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Building applications from individual components that each perform a discrete function lets you scale and change applications quickly.
          Camunda
          It is an open source platform for workflow and decision automation that brings business users and software developers together.
          See all alternatives
          Decisions about Airflow and Apache Beam
          No stack decisions found
          Interest over time
          Reviews of Airflow and Apache Beam
          No reviews found
          How developers use Airflow and Apache Beam
          Avatar of Eugene Ivanchenko
          Eugene Ivanchenko uses AirflowAirflow

          Manage the calculation pipeline and data distribution procedures.

          Avatar of Christopher Davison
          Christopher Davison uses AirflowAirflow

          Used for scheduling ETL jobs

          How much does Airflow cost?
          How much does Apache Beam cost?
          Pricing unavailable
          Pricing unavailable
          News about Apache Beam
          More news