Cloudflow vs Dremio

Overview

Dremio

Stacks116

Followers348

Votes8

Cloudflow

Stacks5

Followers13

Votes0

GitHub Stars323

Forks89

Dremio vs Cloudflow: What are the differences?

What is Dremio? Self-service data for everyone. It is a data-as-a-service platform that empowers users to discover, curate, accelerate, and share any data at any time, regardless of location, volume, or structure. Modern data is managed by a wide range of technologies, including relational databases, NoSQL datastores, file systems, Hadoop, and others.

What is Cloudflow? *Streaming Data Pipeline on Kubernetes *. It enables you to quickly develop, orchestrate, and operate distributed streaming applications on Kubernetes. With Cloudflow, streaming applications are comprised of small composable components wired together with schema-based contracts. It can dramatically accelerate streaming application development—reducing the time required to create, package, and deploy—from weeks to hours.

Dremio and Cloudflow can be primarily classified as "Big Data" tools.

Some of the features offered by Dremio are:

Democratize all your data
Make your data engineers more productive
Accelerate your favorite tools

On the other hand, Cloudflow provides the following key features:

Apache Spark, Apache Flink, and Akka Streams
Focus only on business logic, leave the boilerplate to us
We provide all the tooling for going from business logic to a deployable Docker image

Cloudflow is an open source tool with 172 GitHub stars and 50 GitHub forks. Here's a link to Cloudflow's open source repository on GitHub.

Share your Stack

Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.

View Docs

CLI (Node.js)

Manual

Advice on Dremio, Cloudflow

karunakaran

Consultant

Jun 26, 2020

Needs advice

I am trying to build a data lake by pulling data from multiple data sources ( custom-built tools, excel files, CSV files, etc) and use the data lake to generate dashboards.

My question is which is the best tool to do the following:

Create pipelines to ingest the data from multiple sources into the data lake
Help me in aggregating and filtering data available in the data lake.
Create new reports by combining different data elements from the data lake.

I need to use only open-source tools for this activity.

I appreciate your valuable inputs and suggestions. Thanks in Advance.

80.4k views80.4k

Comments

datocrats-org

Jul 29, 2020

Needs adviceon

Amazon EC2

Tableau

PowerBI

We need to perform ETL from several databases into a data warehouse or data lake. We want to

keep raw and transformed data available to users to draft their own queries efficiently
give users the ability to give custom permissions and SSO
move between open-source on-premises development and cloud-based production environments

We want to use inexpensive Amazon EC2 instances only on medium-sized data set 16GB to 32GB feeding into Tableau Server or PowerBI for reporting and data analysis purposes.

319k views319k

Comments

Detailed Comparison

Dremio	Cloudflow
Dremio—the data lake engine, operationalizes your data lake storage and speeds your analytics processes with a high-performance and high-efficiency query engine while also democratizing data access for data scientists and analysts.	It enables you to quickly develop, orchestrate, and operate distributed streaming applications on Kubernetes. With Cloudflow, streaming applications are comprised of small composable components wired together with schema-based contracts. It can dramatically accelerate streaming application development—reducing the time required to create, package, and deploy—from weeks to hours.
Democratize all your data; Make your data engineers more productive; Accelerate your favorite tools; Self service, for everybody	Apache Spark, Apache Flink, and Akka Streams; Focus only on business logic, leave the boilerplate to us; We provide all the tooling for going from business logic to a deployable Docker image; We provide Kubernetes tooling to deploy your distributed system with a single command, and manage durable connections between processing stages; With a Lightbend subscription, you get all the tools you need to provide insights, observability, and lifecycle management for evolving your distributed streaming application
Statistics
GitHub Stars -	GitHub Stars 323
GitHub Forks -	GitHub Forks 89
Stacks 116	Stacks 5
Followers 348	Followers 13
Votes 8	Votes 0
Pros & Cons
Pros 3 Nice GUI to enable more people to work with Data 2 Easier to Deploy 2 Connect NoSQL databases with RDBMS 1 Free Cons 1 Works only on Iceberg structured data	No community feedback yet
Integrations
Amazon S3 Python Tableau Azure Database for PostgreSQL Qlik Sense PowerBI	Kubernetes Apache Spark Akka Apache Flink

What are some alternatives to Dremio, Cloudflow?

Kubernetes

Kubernetes is an open source orchestration system for Docker containers. It handles scheduling onto nodes in a compute cluster and actively manages workloads to ensure that their state matches the users declared intentions.

Rancher

Rancher is an open source container management platform that includes full distributions of Kubernetes, Apache Mesos and Docker Swarm, and makes it simple to operate container clusters on any cloud or infrastructure platform.

Docker Compose

With Compose, you define a multi-container application in a single file, then spin your application up in a single command which does everything that needs to be done to get it running.

Docker Swarm

Swarm serves the standard Docker API, so any tool which already communicates with a Docker daemon can use Swarm to transparently scale to multiple hosts: Dokku, Compose, Krane, Deis, DockerUI, Shipyard, Drone, Jenkins... and, of course, the Docker client itself.

Tutum

Tutum lets developers easily manage and run lightweight, portable, self-sufficient containers from any application. AWS-like control, Heroku-like ease. The same container that a developer builds and tests on a laptop can run at scale in Tutum.

Google BigQuery

Run super-fast, SQL-like queries against terabytes of data in seconds, using the processing power of Google's infrastructure. Load data with ease. Bulk load your data using Google Cloud Storage or stream it in. Easy access. Access BigQuery by using a browser tool, a command-line tool, or by making calls to the BigQuery REST API with client libraries such as Java, PHP or Python.

Portainer

It is a universal container management tool. It works with Kubernetes, Docker, Docker Swarm and Azure ACI. It allows you to manage containers without needing to know platform-specific code.

Apache Spark

Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning.

Amazon Redshift

It is optimized for data sets ranging from a few hundred gigabytes to a petabyte or more and costs less than $1,000 per terabyte per year, a tenth the cost of most traditional data warehousing solutions.

Qubole

Qubole is a cloud based service that makes big data easy for analysts and data engineers.

Related Comparisons

Some of the features offered by Dremio are:

Democratize all your data
Make your data engineers more productive
Accelerate your favorite tools

On the other hand, Cloudflow provides the following key features:

Apache Spark, Apache Flink, and Akka Streams
Focus only on business logic, leave the boilerplate to us
We provide all the tooling for going from business logic to a deployable Docker image

Cloudflow is an open source tool with 172 GitHub stars and 50 GitHub forks. Here's a link to Cloudflow's open source repository on GitHub.

Cloudflow vs Dremio