Need advice about which tool to choose?Ask the StackShare community!

Airflow

938
1.6K
+ 1
94
dbt

114
115
+ 1
1
Add tool
Pros of Airflow
Pros of dbt
  • 36
    Features
  • 11
    Beautiful UI
  • 10
    Task Dependency Management
  • 9
    Extensibility
  • 9
    Cluster of workers
  • 4
    Python
  • 4
    Open source
  • 3
    Complex workflows
  • 2
    Dashboard
  • 2
    K
  • 2
    Custom operators
  • 1
    Good api
  • 1
    Apache project
  • 1
    Easy for SQL programmers to learn

Sign up to add or upvote prosMake informed product decisions

Sign up to add or upvote consMake informed product decisions

- No public GitHub repository available -

What is Airflow?

Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command lines utilities makes performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress and troubleshoot issues when needed.

What is dbt?

dbt - Documentation

Need advice about which tool to choose?Ask the StackShare community!

What companies use Airflow?
What companies use dbt?

Sign up to get full access to all the companiesMake informed product decisions

What tools integrate with Airflow?
What tools integrate with dbt?
What are some alternatives to Airflow and dbt?
Luigi
It is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Apache NiFi
An easy to use, powerful, and reliable system to process and distribute data. It supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic.
Jenkins
In a nutshell Jenkins CI is the leading open-source continuous integration server. Built with Java, it provides over 300 plugins to support building and testing virtually any project.
AWS Step Functions
AWS Step Functions makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Building applications from individual components that each perform a discrete function lets you scale and change applications quickly.
Pachyderm
Pachyderm is an open source MapReduce engine that uses Docker containers for distributed computations.
See all alternatives
Interest over time
News about dbt
More news