What is Stitch?
Who uses Stitch?
Stitch Integrations
Here are some stack decisions, common use cases and reviews by companies and developers who chose Stitch in their tech stack.
Hello, For security and strategic reasons, we are migrating our apps from AWS/Google to a cloud provider with more security certifications and fewer functionalities, named Outscale. So far we have been using Google BigQuery as our data warehouse with ELT workflows (using Stitch and dbt ) and we need to migrate our data ecosystem to this new cloud provider.
We are setting up a Kubernetes cluster in our new cloud provider for our apps. Regarding the data warehouse, it's not clear if there are advantages/inconvenients about setting it up on kubernetes (apart from having to create node groups and tolerations with more ram/cpu). Also, we are not sure what's the best Open source or on-premise tool to use. The main requirement is that data must remain in the secure cluster, and no external entity (especially US) can have access to it. We have a dev cluster/environment and a production cluster/environment on this cloud.
Regarding the actual DWH usage - Today we have ~1.5TB in BigQuery in production. We're going to run our initial rests with ~50-100GB of data for our test cluster - Most of our data comes from other databases, so in most cases, we already have replicated sources somewhere, and there are only a handful of collections whose source is directly in the DWH (such as snapshots, some external data we've fetched at some point, google analytics, etc) and needs appropriate level of replication - We are a team of 30-ish people, we do not have critical needs regarding analytics speed, and we do not need real time. We rebuild our DBT models 2-3 times a day and this usually proves enough
Apart from postgreSQL, I haven't really found open-source or on-premise alternatives for setting up a data warehouse, and running transformations with DBT. There is also the question of data ingestion, I've selected Airbyte and @meltano and I have troubles understanding if one of the 2 is better but Airbytes seems to have a bigger community.
What do you suggest regarding the data warehouse, and the ELT workflows ? - Kubernetes or not kubernetes ? - Postgresql or something else ? if postgre, what are the important configs you'd have in mind ? - Airbyte/DBT or something else.
Stitch's Features
- Connect to your ecosystem of data sources - UI allows you to configure your data pipeline in a way that balances data freshness with cost and production database load
- Replication frequency - Choose full or incremental loads, and determine how often you want them to run - from every minute, to once every 24 hours
- Data selection - Configure exactly what data gets replicated by selecting the tables, fields, collections, and endpoints you want in your warehouse
- API - With the Stitch API, you're free to replicate data from any source. Its REST API supports JSON or Transit, and recognizes your schema based on the data you send.
- Usage dashboard - Access our simple UI to check usage data like the number of rows synced by data source, and how you're pacing toward your monthly row limit
- Email alerts - Receive immediate notifications when Stitch encounters issues like expired credentials, integration updates, or warehouse errors preventing loads
- Warehouse views - By using the freshness data provided by Stitch, you can build a simple audit table to track replication frequency
- Scalable - Highly ScalableStitch handles all data volumes with no data caps, allowing you to grow without the possibility of an ETL failure
- Transform nested JSON - Stitch provides automatic detection and normalization of nested document structures into relational schemas
- Complete historical data - On your first sync, Stitch replicates all available historical data from your database and SaaS tools. No database dump necessary.
