Need advice about which tool to choose?Ask the StackShare community!
OpenTSDB vs TimescaleDB: What are the differences?
Introduction
OpenTSDB and TimescaleDB are two popular database management systems that are used for storing and analyzing time series data. While they both serve the same purpose, there are several key differences between the two.
Data Model: The data model used in OpenTSDB is a fixed schema model, where data is stored in a single table with a predefined set of columns. On the other hand, TimescaleDB uses a flexible schema model, where data can be stored in multiple tables with different columns based on the time series being stored. This allows for greater flexibility and scalability in managing time series data.
Scalability: OpenTSDB uses a sharding-based architecture to achieve scalability. Sharding involves splitting the data across multiple nodes or clusters to improve performance and handle larger data volumes. In contrast, TimescaleDB uses a hypertable-based architecture, which leverages PostgreSQL's scalability features such as partitioning and parallelism to handle large amounts of time series data. This allows TimescaleDB to scale effectively without the need for manual sharding.
Query Language: OpenTSDB uses a custom query language that is specifically designed for time series data analysis. This query language allows users to perform various operations such as aggregations, filtering, and downsampling on time series data. TimescaleDB, on the other hand, leverages SQL as its query language, which provides users with a familiar and powerful language to interact with the database. This makes it easier for users to write complex queries and take advantage of the rich ecosystem of SQL tools and libraries.
Compression and Retention: OpenTSDB provides built-in compression and retention mechanisms for managing time series data. It uses a technique called compaction to compress older data and reduce storage requirements. It also supports automatic data retention based on time or data size. In contrast, TimescaleDB leverages PostgreSQL's built-in compression and retention capabilities. It supports both lossless and lossy compression techniques and provides flexible options for data retention based on time or other custom criteria.
Complex Analytics: TimescaleDB offers a range of advanced analytics capabilities that are not available in OpenTSDB. These include support for window functions, join operations, subqueries, and user-defined functions. It also allows users to create materialized views for faster query performance. These features enable users to perform complex analytical tasks and integrate time series data with other datasets more efficiently.
Ecosystem and Community: OpenTSDB has a relatively smaller ecosystem and community compared to TimescaleDB, which is built on top of PostgreSQL. TimescaleDB benefits from the extensive ecosystem and community support of PostgreSQL, including a wide range of libraries, tools, and integrations. This makes it easier for users to leverage existing resources and integrations when working with TimescaleDB.
In summary, OpenTSDB and TimescaleDB differ in their data model, scalability approach, query language, compression, analytics capabilities, and ecosystem support. These differences make them suitable for different use cases and environments.
We are building an IOT service with heavy write throughput and fewer reads (we need downsampling records). We prefer to have good reliability when comes to data and prefer to have data retention based on policies.
So, we are looking for what is the best underlying DB for ingesting a lot of data and do queries easily
We had a similar challenge. We started with DynamoDB, Timescale, and even InfluxDB and Mongo - to eventually settle with PostgreSQL. Assuming the inbound data pipeline in queued (for example, Kinesis/Kafka -> S3 -> and some Lambda functions), PostgreSQL gave us a We had a similar challenge. We started with DynamoDB, Timescale and even InfluxDB and Mongo - to eventually settle with PostgreSQL. Assuming the inbound data pipeline in queued (for example, Kinesis/Kafka -> S3 -> and some Lambda functions), PostgreSQL gave us better performance by far.
Druid is amazing for this use case and is a cloud-native solution that can be deployed on any cloud infrastructure or on Kubernetes. - Easy to scale horizontally - Column Oriented Database - SQL to query data - Streaming and Batch Ingestion - Native search indexes It has feature to work as TimeSeriesDB, Datawarehouse, and has Time-optimized partitioning.
if you want to find a serverless solution with capability of a lot of storage and SQL kind of capability then google bigquery is the best solution for that.
I chose TimescaleDB because to be the backend system of our production monitoring system. We needed to be able to keep track of multiple high cardinality dimensions.
The drawbacks of this decision are our monitoring system is a bit more ad hoc than it used to (New Relic Insights)
We are combining this with Grafana for display and Telegraf for data collection
Pros of OpenTSDB
Pros of TimescaleDB
- Open source9
- Easy Query Language8
- Time-series data analysis7
- Established postgresql API and support5
- Reliable4
- Paid support for automatic Retention Policy2
- Chunk-based compression2
- Postgres integration2
- High-performance2
- Fast and scalable2
- Case studies1
Sign up to add or upvote prosMake informed product decisions
Cons of OpenTSDB
Cons of TimescaleDB
- Licensing issues when running on managed databases5