What is Dask?
It is a versatile tool that supports a variety of workloads. It is composed of two parts: Dynamic task scheduling optimized for computation. This is similar to Airflow, Luigi, Celery, or Make, but optimized for interactive computational workloads. Big Data collections like parallel arrays, dataframes, and lists that extend common interfaces like NumPy, Pandas, or Python iterators to larger-than-memory or distributed environments. These parallel collections run on top of dynamic task schedulers.
Dask is a tool in the Data Science Tools category of a tech stack.
Who uses Dask?
4 companies reportedly use Dask in their tech stacks, including Data Science, Gitential, and Metron.
4 developers on StackShare have stated that they use Dask.
Python, Pandas, NumPy, OpenRefine, and PySpark are some of the popular tools that integrate with Dask. Here's a list of all 5 tools that integrate with Dask.
Why developers like Dask?
Here’s a list of reasons why companies and developers use Dask
Be the first to leave a pro
- Supports a variety of workloads
- Dynamic task scheduling
- Trivial to set up and run on a laptop in a single process
- Runs resiliently on clusters with 1000s of cores
Dask Alternatives & Comparisons
What are some alternatives to Dask?
See all alternatives
Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning.
Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more.
Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container of generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily integrate with a wide variety of databases.
A free and open-source distribution of the Python and R programming languages for scientific computing, that aims to simplify package management and deployment. Package versions are managed by the package management system conda.
Python-based ecosystem of open-source software for mathematics, science, and engineering. It contains modules for optimization, linear algebra, integration, interpolation, special functions, FFT, signal and image processing, ODE solvers and other tasks common in science and engineering.