Alternatives to dbt logo

Alternatives to dbt

act, Airflow, Looker, Apache Spark, and Slick are the most popular alternatives and competitors to dbt.
102
99
+ 1
1

What is dbt and what are its top alternatives?

dbt - Documentation
dbt is a tool in the Database Tools category of a tech stack.

Top Alternatives to dbt

  • act

    act

    Rather than having to commit/push every time you want test out the changes you are making to your .github/workflows/ files (or for any changes to embedded GitHub actions), you can use this tool to run the actions locally. The environment variables and filesystem are all configured to match what GitHub provides. ...

  • Airflow

    Airflow

    Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command lines utilities makes performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress and troubleshoot issues when needed. ...

  • Looker

    Looker

    We've built a unique data modeling language, connections to today's fastest analytical databases, and a service that you can deploy on any infrastructure, and explore on any device. Plus, we'll help you every step of the way. ...

  • Apache Spark

    Apache Spark

    Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning. ...

  • Slick

    Slick

    It is a modern database query and access library for Scala. It allows you to work with stored data almost as if you were using Scala collections while at the same time giving you full control over when a database access happens and which data is transferred. ...

  • Spring Data

    Spring Data

    It makes it easy to use data access technologies, relational and non-relational databases, map-reduce frameworks, and cloud-based data services. This is an umbrella project which contains many subprojects that are specific to a given database. ...

  • Microsoft SQL Server Management Studio

    Microsoft SQL Server Management Studio

    It is an integrated environment for managing any SQL infrastructure, from SQL Server to Azure SQL Database. It provides tools to configure, monitor, and administer instances of SQL Server and databases. Use it to deploy, monitor, and upgrade the data-tier components used by your applications, as well as build queries and scripts. ...

  • Sequel Pro

    Sequel Pro

    Sequel Pro is a fast, easy-to-use Mac database management application for working with MySQL databases. ...

dbt alternatives & related posts

act logo

act

3
14
0
Run your GitHub Actions locally
3
14
+ 1
0
PROS OF ACT
    No pros available
    CONS OF ACT
      No cons available

      related act posts

      Airflow logo

      Airflow

      894
      1.4K
      86
      A platform to programmaticaly author, schedule and monitor data pipelines, by Airbnb
      894
      1.4K
      + 1
      86

      related Airflow posts

      Shared insights
      on
      Jenkins
      Airflow

      I am looking for an open-source scheduler tool with cross-functional application dependencies. Some of the tasks I am looking to schedule are as follows:

      1. Trigger Matillion ETL loads
      2. Trigger Attunity Replication tasks that have downstream ETL loads
      3. Trigger Golden gate Replication Tasks
      4. Shell scripts, wrappers, file watchers
      5. Event-driven schedules

      I have used Airflow in the past, and I know we need to create DAGs for each pipeline. I am not familiar with Jenkins, but I know it works with configuration without much underlying code. I want to evaluate both and appreciate any advise

      See more

      I am looking for the best tool to orchestrate #ETL workflows in non-Hadoop environments, mainly for regression testing use cases. Would Airflow or Apache NiFi be a good fit for this purpose?

      For example, I want to run an Informatica ETL job and then run an SQL task as a dependency, followed by another task from Jira. What tool is best suited to set up such a pipeline?

      See more
      Looker logo

      Looker

      294
      301
      9
      Pioneering the next generation of BI, data discovery & data analytics
      294
      301
      + 1
      9

      related Looker posts

      Mohan Ramanujam

      We are a consumer mobile app IOS/Android startup. The app is instrumented with branch and Firebase. We use Google BigQuery. We are looking at tools that can support engagement and cohort analysis at an early stage price which we can grow with. Data Studio is the default but it would seem Looker provides more power. We don't have much insight into Amplitude other than the fact it is a popular PM tool. Please provide some insight.

      See more
      Apache Spark logo

      Apache Spark

      2K
      2.1K
      127
      Fast and general engine for large-scale data processing
      2K
      2.1K
      + 1
      127

      related Apache Spark posts

      Eric Colson
      Chief Algorithms Officer at Stitch Fix · | 20 upvotes · 1.6M views

      The algorithms and data infrastructure at Stitch Fix is housed in #AWS. Data acquisition is split between events flowing through Kafka, and periodic snapshots of PostgreSQL DBs. We store data in an Amazon S3 based data warehouse. Apache Spark on Yarn is our tool of choice for data movement and #ETL. Because our storage layer (s3) is decoupled from our processing layer, we are able to scale our compute environment very elastically. We have several semi-permanent, autoscaling Yarn clusters running to serve our data processing needs. While the bulk of our compute infrastructure is dedicated to algorithmic processing, we also implemented Presto for adhoc queries and dashboards.

      Beyond data movement and ETL, most #ML centric jobs (e.g. model training and execution) run in a similarly elastic environment as containers running Python and R code on Amazon EC2 Container Service clusters. The execution of batch jobs on top of ECS is managed by Flotilla, a service we built in house and open sourced (see https://github.com/stitchfix/flotilla-os).

      At Stitch Fix, algorithmic integrations are pervasive across the business. We have dozens of data products actively integrated systems. That requires serving layer that is robust, agile, flexible, and allows for self-service. Models produced on Flotilla are packaged for deployment in production using Khan, another framework we've developed internally. Khan provides our data scientists the ability to quickly productionize those models they've developed with open source frameworks in Python 3 (e.g. PyTorch, sklearn), by automatically packaging them as Docker containers and deploying to Amazon ECS. This provides our data scientist a one-click method of getting from their algorithms to production. We then integrate those deployments into a service mesh, which allows us to A/B test various implementations in our product.

      For more info:

      #DataScience #DataStack #Data

      See more
      Conor Myhrvold
      Tech Brand Mgr, Office of CTO at Uber · | 7 upvotes · 813K views

      Why we built Marmaray, an open source generic data ingestion and dispersal framework and library for Apache Hadoop :

      Built and designed by our Hadoop Platform team, Marmaray is a plug-in-based framework built on top of the Hadoop ecosystem. Users can add support to ingest data from any source and disperse to any sink leveraging the use of Apache Spark . The name, Marmaray, comes from a tunnel in Turkey connecting Europe and Asia. Similarly, we envisioned Marmaray within Uber as a pipeline connecting data from any source to any sink depending on customer preference:

      https://eng.uber.com/marmaray-hadoop-ingestion-open-source/

      (Direct GitHub repo: https://github.com/uber/marmaray Kafka Kafka Manager )

      See more
      Slick logo

      Slick

      8.3K
      317
      0
      Database query and access library for Scala
      8.3K
      317
      + 1
      0
      PROS OF SLICK
        No pros available
        CONS OF SLICK
          No cons available

          related Slick posts

          Spring Data logo

          Spring Data

          284
          203
          0
          Provides a consistent approach to data access – relational, non-relational, map-reduce, and beyond
          284
          203
          + 1
          0
          PROS OF SPRING DATA
            No pros available
            CONS OF SPRING DATA
              No cons available

              related Spring Data posts

              Остап Комплікевич

              I need some advice to choose an engine for generation web pages from the Spring Boot app. Which technology is the best solution today? 1) JSP + JSTL 2) Apache FreeMarker 3) Thymeleaf Or you can suggest even other perspective tools. I am using Spring Boot, Spring Web, Spring Data, Spring Security, PostgreSQL, Apache Tomcat in my project. I have already tried to generate pages using jsp, jstl, and it went well. However, I had huge problems via carrying already created static pages, to jsp format, because of syntax. Thanks.

              See more
              Microsoft SQL Server Management Studio logo

              Microsoft SQL Server Management Studio

              279
              182
              0
              An integrated environment for managing any SQL infrastructure
              279
              182
              + 1
              0
              PROS OF MICROSOFT SQL SERVER MANAGEMENT STUDIO
                No pros available
                CONS OF MICROSOFT SQL SERVER MANAGEMENT STUDIO
                  No cons available

                  related Microsoft SQL Server Management Studio posts

                  Sequel Pro logo

                  Sequel Pro

                  272
                  232
                  66
                  MySQL database management for Mac OS X
                  272
                  232
                  + 1
                  66
                  PROS OF SEQUEL PRO
                  CONS OF SEQUEL PRO
                    No cons available

                    related Sequel Pro posts