We are building an IOT service with heavy write throughput and fewer reads (we need downsampling records). We prefer to have good reliability when comes to data and prefer to have data retention based on policies.
So, we are looking for what is the best underlying DB for ingesting a lot of data and do queries easily
Another option for scalable serverless SQL-type solution would be AWS Aurora serverless, which unlike BigQuery is 100% SQL compatible (you can choose either MySQL or PostgreSQL flavor).
We had a similar challenge. We started with DynamoDB, Timescale, and even InfluxDB and Mongo - to eventually settle with PostgreSQL. Assuming the inbound data pipeline in queued (for example, Kinesis/Kafka -> S3 -> and some Lambda functions), PostgreSQL gave us a We had a similar challenge. We started with DynamoDB, Timescale and even InfluxDB and Mongo - to eventually settle with PostgreSQL. Assuming the inbound data pipeline in queued (for example, Kinesis/Kafka -> S3 -> and some Lambda functions), PostgreSQL gave us better performance by far.
To echo Yaron, don't sell out RDBM systems so fast - with a correctly designed schema, data throughput of a good engine (such as Postgres or MySQL/Maria) can rival the best of the NoSQL branch.
