269k views269k
CommentsAzure Data Factory vs Databricks
Overview
Share your Stack
Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.
CLI (Node.js)
Manual
Advice on Azure Data Factory, Databricks
Detailed Comparison
It is a service designed to allow developers to integrate disparate data sources. It is a platform somewhat like SSIS in the cloud to manage the data you have both on-prem and in the cloud. | Databricks Unified Analytics Platform, from the original creators of Apache Spark™, unifies data science and engineering across the Machine Learning lifecycle from data preparation to experimentation and deployment of ML applications. |
Real-Time Integration;
Parallel Processing;
Data Chunker;
Data Masking;
Proactive Monitoring;
Big Data Processing |
Built on Apache Spark and optimized for performance;
Reliable and Performant Data Lakes;
Interactive Data Science and Collaboration;
Data Pipelines and Workflow Automation;
End-to-End Data Security and Compliance;
Compatible with Common Tools in the Ecosystem;
Unparalled Support by the Leading Committers of Apache Spark |
Statistics | |
GitHub Stars 516 | GitHub Stars - |
GitHub Forks 610 | GitHub Forks - |
Stacks 253 | Stacks 524 |
Followers 484 | Followers 768 |
Votes 0 | Votes 8 |
Pros & Cons | |
No community feedback yet | Pros
|
Integrations | |
What are some alternatives to Azure Data Factory, Databricks?
Google Analytics
Google Analytics lets you measure your advertising ROI as well as track your Flash, video, and social networking sites and applications.
Mixpanel
Mixpanel helps companies build better products through data. With our powerful, self-serve product analytics solution, teams can easily analyze how and why people engage, convert, and retain to improve their user experience.
Apache Spark
Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning.
Piwik
Matomo (formerly Piwik) is a full-featured PHP MySQL software program that you download and install on your own webserver. At the end of the five-minute installation process, you will be given a JavaScript code.
Presto
Distributed SQL Query Engine for Big Data
Amazon Athena
Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run.
Apache Flink
Apache Flink is an open source system for fast and versatile data analytics in clusters. Flink supports batch and streaming analytics, in one system. Analytical programs can be written in concise and elegant APIs in Java and Scala.
lakeFS
It is an open-source data version control system for data lakes. It provides a “Git for data” platform enabling you to implement best practices from software engineering on your data lake, including branching and merging, CI/CD, and production-like dev/test environments.
Druid
Druid is a distributed, column-oriented, real-time analytics data store that is commonly used to power exploratory dashboards in multi-tenant environments. Druid excels as a data warehousing solution for fast aggregate queries on petabyte sized data sets. Druid supports a variety of flexible filters, exact calculations, approximate algorithms, and other useful calculations.
Clicky
Clicky Web Analytics gives bloggers and smaller web sites a more personal understanding of their visitors. Clicky has various features that helps stand it apart from the competition specifically Spy and RSS feeds that allow web site owners to get live information about their visitors.
