Alexey Timin

Many people try to use TimescaleDB to keep a history of unstructured data. I've investigated the approach and done some benchmarks against ReductStore My conclusion, you can use TimescaleDB with blob data but only with objects smaller than 20-30K, and you have to build your own solution for data retention solution based on disk usage.

In many IoT or computer vision applications, we often need to store a history of unstructured data. It is probably a bad idea to write blobs to a SQL database, especially continuously as a time series. However, MongoDB is a NoSql database that allows time series, it still loses ReductStore in performance.

See the link for the numbers.

In many IoT or computer vision applications, we often need to store a history of unstructured data. It is probably a bad idea to write blobs to a SQL database, especially continuously as a time series. However, MongoDB is a NoSql database that allows time series, it still loses ReductStore in performance.

See the link for the numbers.

In many applications, you may need to capture a history of unstructured data. Previously, we built an in-house solution based on Minio and InfluxDB. Apart from the fact that it wasn't an easy development task, we also ran into performance issues with blobs smaller than 100KB over high latency networks. ReductStore is a time series database for blob data designed specifically for edge computing. It uses HTTP to access data over time intervals and shows much better performance for small blobs due to its batching approach.

ReductStore is a time series database designed to manage large amounts of blob data. For this kind of application, performance is critical. We decided to write it in C++. It was a good decision until we started porting the database to new platforms. Last month I was mostly busy with cross-compilation issues and different behavior on different platforms. We decided to switch to Rust because it has a package manager and cross-compilation out of the box. For a smooth migration we are using cxx.rs library and rewriting modules step by step with deep testing intermediate versions. I guess in a few weeks the migration will be done and ReductStore will be 100% Rust.

ReductStore provides an HTTP API to access data. This allows us to easily integrate it into any cloud environment. In addition, you can scale it by spinning up a few instances of the database and load balancing through NGINX. In this tutorial, you will learn you how you can launch two ReductStore instances by using Docker Compose and split them by using subpaths in NGINX.

You can use ReductStore to keep a history of MQTT messages by using its Client SDKs. This can be useful if you use a binary format for your data and it can be recorded in a classical TSDB. You can set a FIFO quota for a bucket in your ReductStore instance so that the database removes old MQTT messages when the limit is reached.

We tried using Minio to store a history of images from a CV camera. Unfortunately, it didn't work well because it didn't provide an API to retrieve data over a time interval, and it didn't do data redaction for old data. ReductStore is designed for such a kind of applications and works well with a few terabytes of historical data.