Druid Could be an amazing solution for your use case, My understanding, and the assumption is you are looking to export your data from MariaDB for Analytical workload. It can be used for time series database as well as a data warehouse and can be scaled horizontally once your data increases. It's pretty easy to set up on any environment (Cloud, Kubernetes, or Self-hosted nix system). Some important features which make it a perfect solution for your use case. 1. It can do streaming ingestion (Kafka, Kinesis) as well as batch ingestion (Files from Local & Cloud Storage or Databases like MySQL, Postgres). In your case MariaDB (which has the same drivers to MySQL) 2. Columnar Database, So you can query just the fields which are required, and that runs your query faster automatically. 3. Druid intelligently partitions data based on time and time-based queries are significantly faster than traditional databases. 4. Scale up or down by just adding or removing servers, and Druid automatically rebalances. Fault-tolerant architecture routes around server failures 5. Gives ana amazing centralized UI to manage data sources, query, tasks.

Hi, Akash,

I wouldn't make this decision without lots more information. Cloud Firestore has a much richer metamodel (document-oriented) than Dynamo (key-value), and Dynamo seems to be particularly restrictive. That is why it is so fast. There are many needs in most applications to get lightning access to the members of a set, one set at a time. Dynamo DB is a great choice. But, social media applications generally need to be able to make long traverses across a graph. While you can make almost any metamodel act like another one, with your own custom layers on top of it, or just by writing a lot more code, it's a long way around to do that with simple key-value sets. It's hard enough to traverse across networks of collections in a document-oriented database. So, if you are moving, I think a graph-oriented database like Amazon Neptune, or, if you might want built-in reasoning, Allegro or Ontotext, would take the least programming, which is where the most cost and bugs can be avoided. Also, managed systems are also less costly in terms of people's time and system errors. It's easier to measure the costs of managed systems, so they are often seen as more costly.

We had a similar challenge. We started with DynamoDB, Timescale, and even InfluxDB and Mongo - to eventually settle with PostgreSQL. Assuming the inbound data pipeline in queued (for example, Kinesis/Kafka -> S3 -> and some Lambda functions), PostgreSQL gave us a We had a similar challenge. We started with DynamoDB, Timescale and even InfluxDB and Mongo - to eventually settle with PostgreSQL. Assuming the inbound data pipeline in queued (for example, Kinesis/Kafka -> S3 -> and some Lambda functions), PostgreSQL gave us better performance by far.

Druid is amazing for this use case and is a cloud-native solution that can be deployed on any cloud infrastructure or on Kubernetes. - Easy to scale horizontally - Column Oriented Database - SQL to query data - Streaming and Batch Ingestion - Native search indexes It has feature to work as TimeSeriesDB, Datawarehouse, and has Time-optimized partitioning.

if you want to find a serverless solution with capability of a lot of storage and SQL kind of capability then google bigquery is the best solution for that.