Overview

Treasure Data

Stacks28

Followers44

Votes5

Amazon Redshift

Stacks1.5K

Followers1.4K

Votes108

Google BigQuery

Stacks1.7K

Followers1.5K

Votes152

Amazon Redshift vs Google BigQuery vs Treasure Data: What are the differences?

Data Warehouse Architecture: Amazon Redshift is based on a traditional shared-nothing MPP architecture, while Google BigQuery utilizes a serverless architecture, and Treasure Data operates on a microservices-based architecture. This difference impacts how the data is stored, processed, and accessed within each platform.
Pricing Structure: Amazon Redshift follows a pay-as-you-go pricing model where users pay for the compute nodes they use. In contrast, Google BigQuery operates on a pay-per-query pricing model, and Treasure Data offers a flexible pricing structure based on the volume of data processed. This difference can significantly impact the cost of using each data warehousing solution.
Integration and Ecosystem: Amazon Redshift integrates seamlessly with other AWS services and has a robust ecosystem of tools and services. Google BigQuery integrates well with Google Cloud Platform services, and Treasure Data offers connectors to various third-party services. The level of integration and extensibility can affect the overall data workflow efficiency and flexibility.
Performance and Scalability: Amazon Redshift is known for its high performance and scalability, especially for complex queries and large datasets. Google BigQuery excels in processing ad-hoc queries and handling massive datasets quickly. Treasure Data focuses on real-time data processing and stream analytics, enabling high scalability for event-driven workloads. Understanding the specific performance and scalability requirements is crucial in choosing the right data warehousing solution.
Security and Compliance: Amazon Redshift offers robust security features, including encryption at rest and in transit, access control, and compliance certifications. Google BigQuery also provides strong security measures and compliance certifications. Treasure Data prioritizes data governance and compliance, offering features like data masking and privacy protection. Ensuring data security and compliance with industry regulations is paramount for all three data warehouse solutions.
Ease of Use and Management: Amazon Redshift offers a user-friendly interface and comprehensive management tools for monitoring and optimizing performance. Google BigQuery is known for its simplicity and ease of use, requiring minimal setup and maintenance. Treasure Data provides a unified platform for managing data pipelines, workflows, and analytics tasks. The level of ease of use and management convenience can impact user productivity and resource allocation for data teams.

In Summary, Amazon Redshift, Google BigQuery, and Treasure Data differ in their data warehouse architecture, pricing structure, integration and ecosystem, performance and scalability, security and compliance features, and ease of use and management capabilities.

🔥 Trending in Databases on StackShare

pgvector Databases

Open-source vector similarity search for Postgres

Try it View Docs Alternatives

Try

189

7

Advice on Treasure Data, Amazon Redshift, Google BigQuery

datocrats-org

Jul 29, 2020

Needs adviceon

Amazon EC2

Tableau

PowerBI

We need to perform ETL from several databases into a data warehouse or data lake. We want to

keep raw and transformed data available to users to draft their own queries efficiently
give users the ability to give custom permissions and SSO
move between open-source on-premises development and cloud-based production environments

We want to use inexpensive Amazon EC2 instances only on medium-sized data set 16GB to 32GB feeding into Tableau Server or PowerBI for reporting and data analysis purposes.

319k views319k

Comments

Julien

CTO at Hawk

Sep 19, 2020

Decided

Cloud Data-warehouse is the centerpiece of modern Data platform. The choice of the most suitable solution is therefore fundamental.

Our benchmark was conducted over BigQuery and Snowflake. These solutions seem to match our goals but they have very different approaches.

BigQuery is notably the only 100% serverless cloud data-warehouse, which requires absolutely NO maintenance: no re-clustering, no compression, no index optimization, no storage management, no performance management. Snowflake requires to set up (paid) reclustering processes, to manage the performance allocated to each profile, etc. We can also mention Redshift, which we have eliminated because this technology requires even more ops operation.

BigQuery can therefore be set up with almost zero cost of human resources. Its on-demand pricing is particularly adapted to small workloads. 0 cost when the solution is not used, only pay for the query you're running. But quickly the use of slots (with monthly or per-minute commitment) will drastically reduce the cost of use. We've reduced by 10 the cost of our nightly batches by using flex slots.

Finally, a major advantage of BigQuery is its almost perfect integration with Google Cloud Platform services: Cloud functions, Dataflow, Data Studio, etc.

BigQuery is still evolving very quickly. The next milestone, BigQuery Omni, will allow to run queries over data stored in an external Cloud platform (Amazon S3 for example). It will be a major breakthrough in the history of cloud data-warehouses. Omni will compensate a weakness of BigQuery: transferring data in near real time from S3 to BQ is not easy today. It was even simpler to implement via Snowflake's Snowpipe solution.

We also plan to use the Machine Learning features built into BigQuery to accelerate our deployment of Data-Science-based projects. An opportunity only offered by the BigQuery solution

193k views193k

Comments

Detailed Comparison

Treasure Data	Amazon Redshift	Google BigQuery
Treasure Data's Big Data as-a-Service cloud platform enables data-driven businesses to focus their precious development resources on their applications, not on mundane, time-consuming integration and operational tasks. The Treasure Data Cloud Data Warehouse service offers an affordable, quick-to-implement and easy-to-use big data option that does not require specialized IT resources, making big data analytics available to the mass market.	It is optimized for data sets ranging from a few hundred gigabytes to a petabyte or more and costs less than $1,000 per terabyte per year, a tenth the cost of most traditional data warehousing solutions.	Run super-fast, SQL-like queries against terabytes of data in seconds, using the processing power of Google's infrastructure. Load data with ease. Bulk load your data using Google Cloud Storage or stream it in. Easy access. Access BigQuery by using a browser tool, a command-line tool, or by making calls to the BigQuery REST API with client libraries such as Java, PHP or Python.
Instant Integration- Using td-agent, you can start importing your data from existing log files, web and packaged applications right away.;Streaming or Batch?- You choose! Our data collection tool, td-agent, enables you to stream or batch your data to the cloud in JSON format.;Secure Upload- The connection between td-agent and the cloud is SSL-encrypted, ensuring secure transfer of your data.;Availability- Our best-in-class, multi-tenant architecture uses Amazon S3 to ensure 24x7 availability and automatic replication.;Columnar Database- Our columnar database not only delivers blinding performance, it also compresses data to 5 to 10 percent of its original size.;Schema Free- Unlike traditional databases – even cloud databases – Treasure Data allows you to change your data schema anytime.;SQL-like Query Language- Query your data using our SQL-like language.;BI Tools Connectivity- Treasure Data allows you to use your existing BI/visualization tools (e.g. JasperSoft, Pentaho, Talend, Indicee, Metric Insights) using our JDBC driver.;Enterprise-level Service and Support;No Lock-in- We provide a one-line command to let you export your data anywhere you choose, whenever you choose.	Optimized for Data Warehousing- It uses columnar storage, data compression, and zone maps to reduce the amount of IO needed to perform queries. Redshift has a massively parallel processing (MPP) architecture, parallelizing and distributing SQL operations to take advantage of all available resources.;Scalable- With a few clicks of the AWS Management Console or a simple API call, you can easily scale the number of nodes in your data warehouse up or down as your performance or capacity needs change.;No Up-Front Costs- You pay only for the resources you provision. You can choose On-Demand pricing with no up-front costs or long-term commitments, or obtain significantly discounted rates with Reserved Instance pricing.;Fault Tolerant- Amazon Redshift has multiple features that enhance the reliability of your data warehouse cluster. All data written to a node in your cluster is automatically replicated to other nodes within the cluster and all data is continuously backed up to Amazon S3.;SQL - Amazon Redshift is a SQL data warehouse and uses industry standard ODBC and JDBC connections and Postgres drivers.;Isolation - Amazon Redshift enables you to configure firewall rules to control network access to your data warehouse cluster.;Encryption – With just a couple of parameter settings, you can set up Amazon Redshift to use SSL to secure data in transit and hardware-acccelerated AES-256 encryption for data at rest.<br>	All behind the scenes- Your queries can execute asynchronously in the background, and can be polled for status.;Import data with ease- Bulk load your data using Google Cloud Storage or stream it in bursts of up to 1,000 rows per second.;Affordable big data- The first Terabyte of data processed each month is free.;The right interface- Separate interfaces for administration and developers will make sure that you have access to the tools you need.
Statistics
Stacks 28	Stacks 1.5K	Stacks 1.7K
Followers 44	Followers 1.4K	Followers 1.5K
Votes 5	Votes 108	Votes 152
Pros & Cons
Pros 2 Scaleability, less overhead 2 Makes it easy to ingest all data from different inputs 1 Responsive to our business requirements, great support	Pros 41 Data Warehousing 27 Scalable 17 SQL 14 Backed by Amazon 5 Encryption	Pros 28 High Performance 25 Easy to use 22 Fully managed service 19 Cheap Pricing 16 Process hundreds of GB in seconds Cons 1 You can't unit test changes in BQ data 0 Sdas
Integrations
Amazon EC2 G Suite Heroku Engine Yard Cloud Red Hat OpenShift cloudControl	SQLite MySQL Oracle PL/SQL	Xplenty Fluentd Looker Chartio

What are some alternatives to Treasure Data, Amazon Redshift, Google BigQuery?

Qubole

Qubole is a cloud based service that makes big data easy for analysts and data engineers.

Amazon EMR

It is used in a variety of applications, including log analysis, data warehousing, machine learning, financial analysis, scientific simulation, and bioinformatics.

Altiscale

we run Apache Hadoop for you. We not only deploy Hadoop, we monitor, manage, fix, and update it for you. Then we take it a step further: We monitor your jobs, notify you when something’s wrong with them, and can help with tuning.

Snowflake

Snowflake eliminates the administration and management demands of traditional data warehouses and big data platforms. Snowflake is a true data warehouse as a service running on Amazon Web Services (AWS)—no infrastructure to manage and no knobs to turn.

Stitch

Stitch is a simple, powerful ETL service built for software developers. Stitch evolved out of RJMetrics, a widely used business intelligence platform. When RJMetrics was acquired by Magento in 2016, Stitch was launched as its own company.

Azure Synapse

It is an analytics service that brings together enterprise data warehousing and Big Data analytics. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources—at scale. It brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate BI and machine learning needs.

Dremio

Dremio—the data lake engine, operationalizes your data lake storage and speeds your analytics processes with a high-performance and high-efficiency query engine while also democratizing data access for data scientists and analysts.

Cloudera Enterprise

Cloudera Enterprise includes CDH, the world’s most popular open source Hadoop-based platform, as well as advanced system management and data management tools plus dedicated support and community advocacy from our world-class team of Hadoop developers and experts.

Airbyte

It is an open-source data integration platform that syncs data from applications, APIs & databases to data warehouses lakes & DBs.

Xplenty

Read and process data from cloud storage sources such as Amazon S3, Rackspace Cloud Files and IBM SoftLayer Object Storage. Once done processing, Xplenty allows you to connect with Amazon Redshift, SAP HANA and Google BigQuery. You can also store processed data back in your favorite relational database, cloud storage or key-value store.