Qubole logo
Prepare, integrate and explore Big Data in the cloud (Hive, MapReduce, Pig, Presto, Spark and Sqoop)
13
20
+ 1
50

What is Qubole?

Qubole is a cloud based service that makes big data easy for analysts and data engineers.
Qubole is a tool in the Big Data as a Service category of a tech stack.

Who uses Qubole?

Companies
4 companies reportedly use Qubole in their tech stacks, including Pinterest, MediaMath, and Snowplow.

Developers
9 developers on StackShare have stated that they use Qubole.

Why developers like Qubole?

Here’s a list of reasons why companies and developers use Qubole
Qubole Reviews

Here are some stack decisions, common use cases and reviews by companies and developers who chose Qubole in their tech stack.

StackShare Editors
StackShare Editors
Qubole
Qubole
Hadoop
Hadoop
Puppet Labs
Puppet Labs

By mid-2014, around the time of the Series F, Pinterest users had already created more than 30 billion Pins, and the company was logging around 20 terabytes of new data daily, with around 10 petabytes of data in S3. To drive personalization for its users, and to empower engineers to build big data applications quickly, the data team built a self-serve Hadoop platform.

To start, they decoupled compute from storage, which meant teams would have to worry less about loading or synchronizing data, allowing existing or future clusters to make use of the data across a single shared file system.

A centralized Hive metastore act as the source of truth. They chose Hive for most of their Hadoop jobs “primarily because the SQL interface is simple and familiar to people across the industry.”

Dependency management takes place across three layers: *** Baked AMIs, which are large slow-loading dependencies pre-loaded on images; **Automated Configurations (Masterless Puppets), which allows Puppet clients to “pull their configuration from S3 and set up a service that’s responsible for keeping S3 configurations in sync with the Puppet master;” and Runtime Staging on S3, which creates a working directory at runtime for each developer that pulls down its dependencies directly from S3.

Finally, they migrated their Hadoop jobs to Qubole, which “supported AWS/S3 and was relatively easy to get started on.”

See more
John Egan
John Egan
at Pinterest · | 1 upvotes · 1.4K views
atPinterestPinterest
Qubole
Qubole

We ultimately migrated our Hadoop jobs to Qubole, a rising player in the Hadoop as a Service space. Given that EMR had become unstable at our scale, we had to quickly move to a provider that played well with AWS (specifically, spot instances) and S3. Qubole supported AWS/S3 and was relatively easy to get started on. After vetting Qubole and comparing its performance against alternatives (including managed clusters), we decided to go with Qubole Qubole

See more

Qubole's Features

  • Intuitive GUI
  • Optimized Hive
  • Improved S3 Performance
  • Auto Scaling
  • Spot Instance Pricing
  • Managed Clusters
  • Cloud Integration
  • Cluster Lifecycle Management

Qubole Alternatives & Comparisons

What are some alternatives to Qubole?
Databricks
Databricks Unified Analytics Platform, from the original creators of Apache Spark™, unifies data science and engineering across the Machine Learning lifecycle from data preparation to experimentation and deployment of ML applications.
Amazon Redshift
Redshift makes it simple and cost-effective to efficiently analyze all your data using your existing business intelligence tools. It is optimized for datasets ranging from a few hundred gigabytes to a petabyte or more and costs less than $1,000 per terabyte per year, a tenth the cost of most traditional data warehousing solutions.
Google BigQuery
Run super-fast, SQL-like queries against terabytes of data in seconds, using the processing power of Google's infrastructure. Load data with ease. Bulk load your data using Google Cloud Storage or stream it in. Easy access. Access BigQuery by using a browser tool, a command-line tool, or by making calls to the BigQuery REST API with client libraries such as Java, PHP or Python.
Amazon EMR
Amazon EMR is used in a variety of applications, including log analysis, web indexing, data warehousing, machine learning, financial analysis, scientific simulation, and bioinformatics. Customers launch millions of Amazon EMR clusters every year.
Snowflake
Snowflake eliminates the administration and management demands of traditional data warehouses and big data platforms. Snowflake is a true data warehouse as a service running on Amazon Web Services (AWS)—no infrastructure to manage and no knobs to turn.
See all alternatives

Qubole's Stats

- No public GitHub repository available -

Qubole's Followers
20 developers follow Qubole to keep up with related blogs and decisions.
Yury Buldakov
Mohamma76685757
fadi assad
Promod Pillai
MohammadAsh15
Nathan McIntyre
jayden11
Daniel David Leybzon
Jason Kilkenny
Abdullah Cetin CAVDAR