Alternatives to Presto logo

Alternatives to Presto

Apache Spark, Stan, Apache Impala, Snowflake, and Apache Drill are the most popular alternatives and competitors to Presto.
394
1K
+ 1
66

What is Presto and what are its top alternatives?

Distributed SQL Query Engine for Big Data
Presto is a tool in the Big Data Tools category of a tech stack.
Presto is an open source tool with GitHub stars and GitHub forks. Here’s a link to Presto's open source repository on GitHub

Top Alternatives to Presto

  • Apache Spark
    Apache Spark

    Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning. ...

  • Stan
    Stan

    A state-of-the-art platform for statistical modeling and high-performance statistical computation. Used for statistical modeling, data analysis, and prediction in the social, biological, and physical sciences, engineering, and business. ...

  • Apache Impala
    Apache Impala

    Impala is a modern, open source, MPP SQL query engine for Apache Hadoop. Impala is shipped by Cloudera, MapR, and Amazon. With Impala, you can query data, whether stored in HDFS or Apache HBase – including SELECT, JOIN, and aggregate functions – in real time. ...

  • Snowflake
    Snowflake

    Snowflake eliminates the administration and management demands of traditional data warehouses and big data platforms. Snowflake is a true data warehouse as a service running on Amazon Web Services (AWS)—no infrastructure to manage and no knobs to turn. ...

  • Apache Drill
    Apache Drill

    Apache Drill is a distributed MPP query layer that supports SQL and alternative query languages against NoSQL and Hadoop data storage systems. It was inspired in part by Google's Dremel. ...

  • Druid
    Druid

    Druid is a distributed, column-oriented, real-time analytics data store that is commonly used to power exploratory dashboards in multi-tenant environments. Druid excels as a data warehousing solution for fast aggregate queries on petabyte sized data sets. Druid supports a variety of flexible filters, exact calculations, approximate algorithms, and other useful calculations. ...

  • MySQL
    MySQL

    The MySQL software delivers a very fast, multi-threaded, multi-user, and robust SQL (Structured Query Language) database server. MySQL Server is intended for mission-critical, heavy-load production systems as well as for embedding into mass-deployed software. ...

  • PostgreSQL
    PostgreSQL

    PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions. ...

Presto alternatives & related posts

Apache Spark logo

Apache Spark

3K
3.5K
140
Fast and general engine for large-scale data processing
3K
3.5K
+ 1
140
PROS OF APACHE SPARK
  • 61
    Open-source
  • 48
    Fast and Flexible
  • 8
    One platform for every big data problem
  • 8
    Great for distributed SQL like applications
  • 6
    Easy to install and to use
  • 3
    Works well for most Datascience usecases
  • 2
    Interactive Query
  • 2
    Machine learning libratimery, Streaming in real
  • 2
    In memory Computation
CONS OF APACHE SPARK
  • 4
    Speed

related Apache Spark posts

Eric Colson
Chief Algorithms Officer at Stitch Fix · | 21 upvotes · 6.1M views

The algorithms and data infrastructure at Stitch Fix is housed in #AWS. Data acquisition is split between events flowing through Kafka, and periodic snapshots of PostgreSQL DBs. We store data in an Amazon S3 based data warehouse. Apache Spark on Yarn is our tool of choice for data movement and #ETL. Because our storage layer (s3) is decoupled from our processing layer, we are able to scale our compute environment very elastically. We have several semi-permanent, autoscaling Yarn clusters running to serve our data processing needs. While the bulk of our compute infrastructure is dedicated to algorithmic processing, we also implemented Presto for adhoc queries and dashboards.

Beyond data movement and ETL, most #ML centric jobs (e.g. model training and execution) run in a similarly elastic environment as containers running Python and R code on Amazon EC2 Container Service clusters. The execution of batch jobs on top of ECS is managed by Flotilla, a service we built in house and open sourced (see https://github.com/stitchfix/flotilla-os).

At Stitch Fix, algorithmic integrations are pervasive across the business. We have dozens of data products actively integrated systems. That requires serving layer that is robust, agile, flexible, and allows for self-service. Models produced on Flotilla are packaged for deployment in production using Khan, another framework we've developed internally. Khan provides our data scientists the ability to quickly productionize those models they've developed with open source frameworks in Python 3 (e.g. PyTorch, sklearn), by automatically packaging them as Docker containers and deploying to Amazon ECS. This provides our data scientist a one-click method of getting from their algorithms to production. We then integrate those deployments into a service mesh, which allows us to A/B test various implementations in our product.

For more info:

#DataScience #DataStack #Data

See more
Patrick Sun
Software Engineer at Stitch Fix · | 10 upvotes · 52.8K views

As a frontend engineer on the Algorithms & Analytics team at Stitch Fix, I work with data scientists to develop applications and visualizations to help our internal business partners make data-driven decisions. I envisioned a platform that would assist data scientists in the data exploration process, allowing them to visually explore and rapidly iterate through their assumptions, then share their insights with others. This would align with our team's philosophy of having engineers "deploy platforms, services, abstractions, and frameworks that allow the data scientists to conceive of, develop, and deploy their ideas with autonomy", and solve the pain of data exploration.

The final product, code-named Dora, is built with React, Redux.js and Victory, backed by Elasticsearch to enable fast and iterative data exploration, and uses Apache Spark to move data from our Amazon S3 data warehouse into the Elasticsearch cluster.

See more
Stan logo

Stan

64
27
0
A Probabilistic Programming Language
64
27
+ 1
0
PROS OF STAN
    Be the first to leave a pro
    CONS OF STAN
      Be the first to leave a con

      related Stan posts

      Apache Impala logo

      Apache Impala

      145
      301
      18
      Real-time Query for Hadoop
      145
      301
      + 1
      18
      PROS OF APACHE IMPALA
      • 11
        Super fast
      • 1
        Massively Parallel Processing
      • 1
        Load Balancing
      • 1
        Replication
      • 1
        Scalability
      • 1
        Distributed
      • 1
        High Performance
      • 1
        Open Sourse
      CONS OF APACHE IMPALA
        Be the first to leave a con

        related Apache Impala posts

        I have been working on a Java application to demonstrate the latency for the select/insert/update operations on KUDU storage using Apache Kudu API - Java based client. I have a few queries about using Apache Kudu API

        1. Do we have JDBC wrapper to use Apache Kudu API for getting connection to Kudu masters with connection pool mechanism and all DB operations?

        2. Does Apache KuduAPI supports order by, group by, and aggregate functions? if yes, how to implement these functions using Kudu APIs.

        3. How can we add kudu predicates to Kudu update operation? if yes, how?

        4. Does Apache Kudu API supports batch insertion (execute the Kudu Insert for multiple rows at one go instead of row by row)? (like Kudusession.apply(List);)

        5. Does Apache Kudu API support join on tables?

        6. which tool is preferred over others (Apache Impala /Kudu API) for read and update/insert DB operations?

        See more
        Snowflake logo

        Snowflake

        1.1K
        1.2K
        27
        The data warehouse built for the cloud
        1.1K
        1.2K
        + 1
        27
        PROS OF SNOWFLAKE
        • 7
          Public and Private Data Sharing
        • 4
          Multicloud
        • 4
          Good Performance
        • 4
          User Friendly
        • 3
          Great Documentation
        • 2
          Serverless
        • 1
          Economical
        • 1
          Usage based billing
        • 1
          Innovative
        CONS OF SNOWFLAKE
          Be the first to leave a con

          related Snowflake posts

          I'm wondering if any Cloud Firestore users might be open to sharing some input and challenges encountered when trying to create a low-cost, low-latency data pipeline to their Analytics warehouse (e.g. Google BigQuery, Snowflake, etc...)

          I'm working with a platform by the name of Estuary.dev, an ETL/ELT and we are conducting some research on the pain points here to see if there are drawbacks of the Firestore->BQ extension and/or if users are seeking easy ways for getting nosql->fine-grained tabular data

          Please feel free to drop some knowledge/wish list stuff on me for a better pipeline here!

          See more
          Shared insights
          on
          Google BigQueryGoogle BigQuerySnowflakeSnowflake

          I use Google BigQuery because it makes is super easy to query and store data for analytics workloads. If you're using GCP, you're likely using BigQuery. However, running data viz tools directly connected to BigQuery will run pretty slow. They recently announced BI Engine which will hopefully compete well against big players like Snowflake when it comes to concurrency.

          What's nice too is that it has SQL-based ML tools, and it has great GIS support!

          See more
          Apache Drill logo

          Apache Drill

          71
          170
          16
          Schema-Free SQL Query Engine for Hadoop and NoSQL
          71
          170
          + 1
          16
          PROS OF APACHE DRILL
          • 4
            NoSQL and Hadoop
          • 3
            Free
          • 3
            Lightning speed and simplicity in face of data jungle
          • 2
            Well documented for fast install
          • 1
            SQL interface to multiple datasources
          • 1
            Nested Data support
          • 1
            Read Structured and unstructured data
          • 1
            V1.10 released - https://drill.apache.org/
          CONS OF APACHE DRILL
            Be the first to leave a con

            related Apache Drill posts

            Druid logo

            Druid

            380
            867
            32
            Fast column-oriented distributed data store
            380
            867
            + 1
            32
            PROS OF DRUID
            • 15
              Real Time Aggregations
            • 6
              Batch and Real-Time Ingestion
            • 5
              OLAP
            • 3
              OLAP + OLTP
            • 2
              Combining stream and historical analytics
            • 1
              OLTP
            CONS OF DRUID
            • 3
              Limited sql support
            • 2
              Joins are not supported well
            • 1
              Complexity

            related Druid posts

            Shared insights
            on
            DruidDruidMongoDBMongoDB

            My background is in Data analytics in the telecom domain. Have to build the database for analyzing large volumes of CDR data so far the data are maintained in a file server and the application queries data from the files. It's consuming a lot of resources queries are taking time so now I am asked to come up with the approach. I planned to rewrite the app, so which database needs to be used. I am confused between MongoDB and Druid.

            So please do advise me on picking from these two and why?

            See more

            My process is like this: I would get data once a month, either from Google BigQuery or as parquet files from Azure Blob Storage. I have a script that does some cleaning and then stores the result as partitioned parquet files because the following process cannot handle loading all data to memory.

            The next process is making a heavy computation in a parallel fashion (per partition), and storing 3 intermediate versions as parquet files: two used for statistics, and the third will be filtered and create the final files.

            I make a report based on the two files in Jupyter notebook and convert it to HTML.

            • Everything is done with vanilla python and Pandas.
            • sometimes I may get a different format of data
            • cloud service is Microsoft Azure.

            What I'm considering is the following:

            Get the data with Kafka or with native python, do the first processing, and store data in Druid, the second processing will be done with Apache Spark getting data from apache druid.

            the intermediate states can be stored in druid too. and visualization would be with apache superset.

            See more
            MySQL logo

            MySQL

            124.8K
            105.6K
            3.8K
            The world's most popular open source database
            124.8K
            105.6K
            + 1
            3.8K
            PROS OF MYSQL
            • 800
              Sql
            • 679
              Free
            • 562
              Easy
            • 528
              Widely used
            • 490
              Open source
            • 180
              High availability
            • 160
              Cross-platform support
            • 104
              Great community
            • 79
              Secure
            • 75
              Full-text indexing and searching
            • 26
              Fast, open, available
            • 16
              Reliable
            • 16
              SSL support
            • 15
              Robust
            • 9
              Enterprise Version
            • 7
              Easy to set up on all platforms
            • 3
              NoSQL access to JSON data type
            • 1
              Relational database
            • 1
              Easy, light, scalable
            • 1
              Sequel Pro (best SQL GUI)
            • 1
              Replica Support
            CONS OF MYSQL
            • 16
              Owned by a company with their own agenda
            • 3
              Can't roll back schema changes

            related MySQL posts

            Nick Rockwell
            SVP, Engineering at Fastly · | 46 upvotes · 3.9M views

            When I joined NYT there was already broad dissatisfaction with the LAMP (Linux Apache HTTP Server MySQL PHP) Stack and the front end framework, in particular. So, I wasn't passing judgment on it. I mean, LAMP's fine, you can do good work in LAMP. It's a little dated at this point, but it's not ... I didn't want to rip it out for its own sake, but everyone else was like, "We don't like this, it's really inflexible." And I remember from being outside the company when that was called MIT FIVE when it had launched. And been observing it from the outside, and I was like, you guys took so long to do that and you did it so carefully, and yet you're not happy with your decisions. Why is that? That was more the impetus. If we're going to do this again, how are we going to do it in a way that we're gonna get a better result?

            So we're moving quickly away from LAMP, I would say. So, right now, the new front end is React based and using Apollo. And we've been in a long, protracted, gradual rollout of the core experiences.

            React is now talking to GraphQL as a primary API. There's a Node.js back end, to the front end, which is mainly for server-side rendering, as well.

            Behind there, the main repository for the GraphQL server is a big table repository, that we call Bodega because it's a convenience store. And that reads off of a Kafka pipeline.

            See more
            Tim Abbott

            We've been using PostgreSQL since the very early days of Zulip, but we actually didn't use it from the beginning. Zulip started out as a MySQL project back in 2012, because we'd heard it was a good choice for a startup with a wide community. However, we found that even though we were using the Django ORM for most of our database access, we spent a lot of time fighting with MySQL. Issues ranged from bad collation defaults, to bad query plans which required a lot of manual query tweaks.

            We ended up getting so frustrated that we tried out PostgresQL, and the results were fantastic. We didn't have to do any real customization (just some tuning settings for how big a server we had), and all of our most important queries were faster out of the box. As a result, we were able to delete a bunch of custom queries escaping the ORM that we'd written to make the MySQL query planner happy (because postgres just did the right thing automatically).

            And then after that, we've just gotten a ton of value out of postgres. We use its excellent built-in full-text search, which has helped us avoid needing to bring in a tool like Elasticsearch, and we've really enjoyed features like its partial indexes, which saved us a lot of work adding unnecessary extra tables to get good performance for things like our "unread messages" and "starred messages" indexes.

            I can't recommend it highly enough.

            See more
            PostgreSQL logo

            PostgreSQL

            97.8K
            81.9K
            3.5K
            A powerful, open source object-relational database system
            97.8K
            81.9K
            + 1
            3.5K
            PROS OF POSTGRESQL
            • 763
              Relational database
            • 510
              High availability
            • 439
              Enterprise class database
            • 383
              Sql
            • 304
              Sql + nosql
            • 173
              Great community
            • 147
              Easy to setup
            • 131
              Heroku
            • 130
              Secure by default
            • 113
              Postgis
            • 50
              Supports Key-Value
            • 48
              Great JSON support
            • 34
              Cross platform
            • 33
              Extensible
            • 28
              Replication
            • 26
              Triggers
            • 23
              Multiversion concurrency control
            • 23
              Rollback
            • 21
              Open source
            • 18
              Heroku Add-on
            • 17
              Stable, Simple and Good Performance
            • 15
              Powerful
            • 13
              Lets be serious, what other SQL DB would you go for?
            • 11
              Good documentation
            • 9
              Scalable
            • 8
              Free
            • 8
              Reliable
            • 8
              Intelligent optimizer
            • 7
              Transactional DDL
            • 7
              Modern
            • 6
              One stop solution for all things sql no matter the os
            • 5
              Relational database with MVCC
            • 5
              Faster Development
            • 4
              Full-Text Search
            • 4
              Developer friendly
            • 3
              Excellent source code
            • 3
              Free version
            • 3
              Great DB for Transactional system or Application
            • 3
              Relational datanbase
            • 3
              search
            • 3
              Open-source
            • 2
              Text
            • 2
              Full-text
            • 1
              Can handle up to petabytes worth of size
            • 1
              Composability
            • 1
              Multiple procedural languages supported
            • 0
              Native
            CONS OF POSTGRESQL
            • 10
              Table/index bloatings

            related PostgreSQL posts

            Simon Reymann
            Senior Fullstack Developer at QUANTUSflow Software GmbH · | 30 upvotes · 10.7M views

            Our whole DevOps stack consists of the following tools:

            • GitHub (incl. GitHub Pages/Markdown for Documentation, GettingStarted and HowTo's) for collaborative review and code management tool
            • Respectively Git as revision control system
            • SourceTree as Git GUI
            • Visual Studio Code as IDE
            • CircleCI for continuous integration (automatize development process)
            • Prettier / TSLint / ESLint as code linter
            • SonarQube as quality gate
            • Docker as container management (incl. Docker Compose for multi-container application management)
            • VirtualBox for operating system simulation tests
            • Kubernetes as cluster management for docker containers
            • Heroku for deploying in test environments
            • nginx as web server (preferably used as facade server in production environment)
            • SSLMate (using OpenSSL) for certificate management
            • Amazon EC2 (incl. Amazon S3) for deploying in stage (production-like) and production environments
            • PostgreSQL as preferred database system
            • Redis as preferred in-memory database/store (great for caching)

            The main reason we have chosen Kubernetes over Docker Swarm is related to the following artifacts:

            • Key features: Easy and flexible installation, Clear dashboard, Great scaling operations, Monitoring is an integral part, Great load balancing concepts, Monitors the condition and ensures compensation in the event of failure.
            • Applications: An application can be deployed using a combination of pods, deployments, and services (or micro-services).
            • Functionality: Kubernetes as a complex installation and setup process, but it not as limited as Docker Swarm.
            • Monitoring: It supports multiple versions of logging and monitoring when the services are deployed within the cluster (Elasticsearch/Kibana (ELK), Heapster/Grafana, Sysdig cloud integration).
            • Scalability: All-in-one framework for distributed systems.
            • Other Benefits: Kubernetes is backed by the Cloud Native Computing Foundation (CNCF), huge community among container orchestration tools, it is an open source and modular tool that works with any OS.
            See more
            Jeyabalaji Subramanian

            Recently we were looking at a few robust and cost-effective ways of replicating the data that resides in our production MongoDB to a PostgreSQL database for data warehousing and business intelligence.

            We set ourselves the following criteria for the optimal tool that would do this job: - The data replication must be near real-time, yet it should NOT impact the production database - The data replication must be horizontally scalable (based on the load), asynchronous & crash-resilient

            Based on the above criteria, we selected the following tools to perform the end to end data replication:

            We chose MongoDB Stitch for picking up the changes in the source database. It is the serverless platform from MongoDB. One of the services offered by MongoDB Stitch is Stitch Triggers. Using stitch triggers, you can execute a serverless function (in Node.js) in real time in response to changes in the database. When there are a lot of database changes, Stitch automatically "feeds forward" these changes through an asynchronous queue.

            We chose Amazon SQS as the pipe / message backbone for communicating the changes from MongoDB to our own replication service. Interestingly enough, MongoDB stitch offers integration with AWS services.

            In the Node.js function, we wrote minimal functionality to communicate the database changes (insert / update / delete / replace) to Amazon SQS.

            Next we wrote a minimal micro-service in Python to listen to the message events on SQS, pickup the data payload & mirror the DB changes on to the target Data warehouse. We implemented source data to target data translation by modelling target table structures through SQLAlchemy . We deployed this micro-service as AWS Lambda with Zappa. With Zappa, deploying your services as event-driven & horizontally scalable Lambda service is dumb-easy.

            In the end, we got to implement a highly scalable near realtime Change Data Replication service that "works" and deployed to production in a matter of few days!

            See more