AWS Data Pipeline vs Azure Pipelines

Get Advice Icon

Need advice about which tool to choose?Ask the StackShare community!

AWS Data Pipeline
AWS Data Pipeline

55
88
+ 1
1
Azure Pipelines
Azure Pipelines

76
66
+ 1
2
Add tool
- No public GitHub repository available -
- No public GitHub repository available -

What is AWS Data Pipeline?

AWS Data Pipeline is a web service that provides a simple management system for data-driven workflows. Using AWS Data Pipeline, you define a pipeline composed of the “data sources” that contain your data, the “activities” or business logic such as EMR jobs or SQL queries, and the “schedule” on which your business logic executes. For example, you could define a job that, every hour, runs an Amazon Elastic MapReduce (Amazon EMR)–based analysis on that hour’s Amazon Simple Storage Service (Amazon S3) log data, loads the results into a relational database for future lookup, and then automatically sends you a daily summary email.

What is Azure Pipelines?

Fast builds with parallel jobs and test execution. Use container jobs to create consistent and reliable builds with the exact tools you need. Create new containers with ease and push them to any registry.
Get Advice Icon

Need advice about which tool to choose?Ask the StackShare community!

Why do developers choose AWS Data Pipeline?
Why do developers choose Azure Pipelines?
    Be the first to leave a con
      Be the first to leave a con
      Jobs that mention AWS Data Pipeline and Azure Pipelines as a desired skillset
      What companies use AWS Data Pipeline?
      What companies use Azure Pipelines?

      Sign up to get full access to all the companiesMake informed product decisions

      What tools integrate with AWS Data Pipeline?
      What tools integrate with Azure Pipelines?

      Sign up to get full access to all the tool integrationsMake informed product decisions

      What are some alternatives to AWS Data Pipeline and Azure Pipelines?
      AWS Glue
      A fully managed extract, transform, and load (ETL) service that makes it easy for customers to prepare and load their data for analytics.
      Airflow
      Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command lines utilities makes performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress and troubleshoot issues when needed.
      AWS Step Functions
      AWS Step Functions makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Building applications from individual components that each perform a discrete function lets you scale and change applications quickly.
      Apache NiFi
      An easy to use, powerful, and reliable system to process and distribute data. It supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic.
      AWS Batch
      It enables developers, scientists, and engineers to easily and efficiently run hundreds of thousands of batch computing jobs on AWS. It dynamically provisions the optimal quantity and type of compute resources (e.g., CPU or memory optimized instances) based on the volume and specific resource requirements of the batch jobs submitted.
      See all alternatives
      Decisions about AWS Data Pipeline and Azure Pipelines
      Oliver Burn
      Oliver Burn
      Architect at Atlassian · | 12 upvotes · 180.3K views
      atAtlassianAtlassian
      Jira
      Jira
      Bitbucket
      Bitbucket
      CircleCI
      CircleCI
      AWS CodePipeline
      AWS CodePipeline
      Octopus Deploy
      Octopus Deploy
      jFrog
      jFrog
      Azure Pipelines
      Azure Pipelines

      We recently added new APIs to Jira to associate information about Builds and Deployments to Jira issues.

      The new APIs were developed using a spec-first API approach for speed and sanity. The details of this approach are described in this blog post, and we relied on using Swagger and associated tools like Swagger UI.

      A new service was created for managing the data. It provides a REST API for external use, and an internal API based on GraphQL. The service is built using Kotlin for increased developer productivity and happiness, and the Spring-Boot framework. PostgreSQL was chosen for the persistence layer, as we have non-trivial requirements that cannot be easily implemented on top of a key-value store.

      The front-end has been built using React and querying the back-end service using an internal GraphQL API. We have plans of providing a public GraphQL API in the future.

      New Jira Integrations: Bitbucket CircleCI AWS CodePipeline Octopus Deploy jFrog Azure Pipelines

      See more
      Interest over time
      Reviews of AWS Data Pipeline and Azure Pipelines
      No reviews found
      How developers use AWS Data Pipeline and Azure Pipelines
      No items found
      How much does AWS Data Pipeline cost?
      How much does Azure Pipelines cost?
      Pricing unavailable
      News about AWS Data Pipeline
      More news
      News about Azure Pipelines
      More news