Hi everybody,
We are building a prototype of our first Hardware system. It's a sensor that we expect to send data approx. every 5 secs. If we scale it to about 1k sensors, it would mean 173M records per month.
We will offer data-retention for 7 days or 1 month based on the plan, but if by any chance the project goes well I see that we can have a lot of pain if not choosing the persistence layer correctly from day 1. Which data storage would you use for that?
Cheers, Alberto.
We have seen really good performance with frequent writes in Redis (~10k writes per second). The retrievals are also fast. I see no reason why it should not be your first choice. I know Redis doesn't have the image of a persistence database but it is capable of persistence and high availability. Moreover, MongoDB or some other database can be used as a cold storage. Even switching to MongoDB or any other NoSQL database entirely wouldn't be a big problem since Redis is a simple key-value store.
Hasura is a full platform working with Postgres and GraphQL. With Hasura you get everything and it is fast and cheap to scale up. With a good infrastructure like Kubernetes or AWS ECS you can run whatever you want.
MongoDB is nice but with millions of rows tends to lose its charm and with MongoDB or Redis you must to develop some backend with Hasura you get a backend ready to use.
