Need advice about which tool to choose?Ask the StackShare community!

OpenTSDB

33
74
+ 1
0
SQLite

18.6K
14.6K
+ 1
535
Add tool

OpenTSDB vs SQLite: What are the differences?

Introduction

OpenTSDB and SQLite are both database systems used in different contexts. OpenTSDB is a scalable distributed time-series database system designed for handling large amounts of time-series data, typically used in monitoring and analytics applications. SQLite, on the other hand, is a lightweight embedded database engine that provides a self-contained, serverless, zero-configuration SQL database, commonly used in applications that require a local database storage.

  1. Storage Model: OpenTSDB stores data based on a unique timestamp and key-value pair, where the key represents a metric and tags represent additional dimensions. SQLite, on the other hand, follows a traditional table-based storage model with rows and columns, suitable for structured data storage and query operations.

  2. Scalability and Distribution: OpenTSDB is designed to handle large scale data and can be distributed across multiple nodes for increased performance and capacity. SQLite, being an embedded database, is more suitable for single-node deployments and applications with moderate data volumes.

  3. Query Language: OpenTSDB uses a custom query language optimized for time-series data analysis, with support for range and aggregation queries over time intervals. SQLite, on the other hand, uses standard SQL and supports a wide range of complex query operations, including joins, subqueries, and other SQL features.

  4. Concurrency and Transactions: OpenTSDB provides limited support for multi-threaded access and limited transactions due to its focus on write-heavy workloads. SQLite, on the other hand, provides robust concurrency control and ACID-compliant transaction support, making it suitable for concurrent read and write operations.

  5. Deployment and Resource Requirements: OpenTSDB requires a distributed cluster setup with multiple nodes for scalability, involving additional configuration and management overhead. SQLite, being an embedded database, has minimal deployment requirements and is very resource-efficient, making it suitable for low-resource environments or applications that need a simple and lightweight data storage solution.

  6. Community and Ecosystem: OpenTSDB has a specific focus on time-series data processing and has a smaller but active community with dedicated support for monitoring and analytics use cases. SQLite, being a widely used embedded database, has a large and mature community with extensive support, tools, and third-party libraries available for various application domains.

In Summary, OpenTSDB is designed for handling large-scale time-series data with a unique storage model and custom query language, while SQLite is a lightweight embedded database with traditional table-based storage and full SQL query capabilities.

Advice on OpenTSDB and SQLite
Needs advice
on
FirebaseFirebaseMySQLMySQL
and
SQLiteSQLite

Hi everyone! I am a high school student, starting a massive project. I'm building a system for a boarding school to be better connected to their students and be more efficient with information. In the meantime, I am developing a website and an android app. What's the best datastore I can use? I need to be able to access student data on the app from the main database and send push notifications. Also feed updates. What's the best approach? What's the best tool I can use to deploy the website and the database? One for testing and prototyping, and an official one... Thanks in advance!!!!

See more
Replies (3)
Ahmed AlAskalany
Android Developer at Kitab Sawti · | 5 upvotes · 304.1K views
Recommends
on
FirebaseFirebase

Firebase has Android, iOS, and Web SDKs; and a console where you can develop, manage, and monitor all the data and analytics from one place. Firebase real-time database is good for online presence and instant feed updates, while Firebase Firestone is good for user profile and other relational data records. Firebase has a UI SDK which makes it easy to interface with the resources in the project, and with tons of tutorials and starter projects it should be easy to quickly have a decent prototype to iterate upon. Since you said Massive, use their pricing calculator to figure if your expected scale will be covered by the free quota or if you go for the pay-as-you-go that the price is reasonable for your project.

Good luck with the project!

See more
Paul Whittemore
Developer and Owner at Appurist Software · | 4 upvotes · 304.2K views
Recommends
on
FirebaseFirebase

It sounds like a server-client relationship (central database) and while SQLite is probably the simplest, note that its performance is probably the worst of the top 20 or so choices you have. It is different from Firebase and MySQL (and most other databases) in that it is embedded in the product, although it could be embedded in your server itself.

MySQL would require a separate MySQL db server, which means either two servers (one for MySQL, and one to provide your specific services to your client app) or both running on a single server machine. There are many alternatives in the same category as MySQL, and a choice of relational databases or document (NoSQL) databases. But architecturally, they are in the same category as MySQL, a separate db server that your application server would get its data from.

Firebase is different yet again, in that it is a service that is already hosted by a company, providing many integrated features such as authentication and storage of user account info. However it does take care of many of the concerns with running a server, such as performance, scalability and management. There are some negatives that you should be aware of though: any investment of time and coding with Firebase is pretty much non-portable, in that you are stuck with Firebase going forward. If you needed to switch to a different service, not only would it be a different API, but it would be a different architecture and much of your coding would need to be discarded. Second, it's owned and run by Google now, so you have a large corporation backing it, but that also means they could decide to discontinue it without any real effect on the Google bottom line. Also some folks would have concerns with storing data on Google servers. That said, I think if you are aware of these in advance, and especially if you are a high school student, that Firebase is a fairly easy winner here. The server is already set up for you, the documentation is very complete and rich, with lots of examples, and Google is not going away. The main concern would be if it really is massive, there could be a rising cost to the service. I suspect though that it is not massive, even if everyone in a school used it. The number of concurrent connections would not be huge (probably not even into the hundreds, even if there are thousands of users).

I'd go with Firebase even though you will need to learn their API, because you'll need to learn something one way or another. SQLite is a bit of a toy database, and MySQL is a real one but you (or someone) would need to manage that server on top of needing to develop the server and client app. With Firebase, much of the server already exists, including a professionally hosted database. There are tons of high-level features provided and initial cost is somewhere between very low and zero.

Part of this is dependent on what language you want to write this in. Javascript for a cross-platform client app (I'd use Vue.js + Vuetify for UI, and provide it as a web app and optionally wrap that with Electron for a desktop app, Apache Cordova for mobile). Server could be Javascript with an Express-based REST API on Node.js, talking to Firebase for services.

If you were a Java developer though, all this goes out the window and I'd recommend a simple Java server with Javalin for REST API, and embedded ObjectDB for database storage (combined into one server). ObjectDB is very very fast and can be separated out into a scalable server if this became truly massive. But you would probably never need to go that far.

All of this is a lot of work. I hope this isn't for something like an assignment. It is in the order of 6 months of work if you know what you're doing, all year if you're learning as you go.

See more
Michael Maraist
Chief Architect at Pixia Corp · | 2 upvotes · 303.5K views
Recommends
on
RocksDBRocksDB

Don't think you can go wrong with MySQL or postgresql. python+postgres is VERY well supported stack and can do almost anything. Great visualization and administrative tools for both. There are some data-mismatch problems, however.. node.js/python with mongodb is a bit more modern and makes it trivial to "serialize" data with sprinklings of indexes. If you're using go-lang, then RocksDB is a great high-performance data-modeling base (it's not relational how-ever) It's more like a building-block for key-value store. But it's ACID so you CAN build relational systems on top. I've used LevelDB for other projects (Java/C) (similar architecture and works great on android - chrome uses it for it's metadata-storage). Rock/Level can achieve multi-million writes on cheap hardware thanks to it's trade-offs.

I'm very familiar with SQLite.. Personally my least favorite, but it's the most portable database format, and it does support ACID.. I have many gripes, but biggest issue is parallel access (you really need a single process/thread to own the data-model, then use IPC to communicate with your process/thread).. (same could be said for LevelDB, but that's so efficient, it's almost never an issue).

If your'e using Java, then JavaDB/DerbyDB/HSQLDB are EXCELLENT systems.. highly multi-threaded, good stand-alone tools. (embedded or TCP-connected). Perfect for unit-tests. Can use simple dumb portable formats (e.g. text-file containing only inserts) all the way to classic journaled binary B-tree formats to pure-in-memory. Java has a lot of overhead, so this is only really viable if you're already using Java in your project.

For high performance "memsql" is mysql API to a hybrid in-memory index + on-disk column-database (feels like classic SQL to you though). Falls into the mysql-swiss-army-knife tool-kit.

Similarly with in-memory there is "redis".. Absolutely a joy to work with. It too is a specialty swiss army knife. Steer clear of redis for primary data that you can't lose.. while redis does support persisting data, it isn't very efficient and will become the bottleneck. redis is great for micro-queue's, topics, stat-aggregators, message-repositories (password-management systems, where writes are rare so persistance is viable). Plus I love that redis uses a pure-text protocol so I can netcat or telnet directly into it and do stuff.

I've loved cloud-data-stores.. Amazon "DynamoDB" or Google BigTable are awesome!!! Cheap compared to normal hosting fees of an AWS EC2 instance.. You can play all day.. put a terabyte up, then blow it away.. pay for what you play with. It's a very very different data-model though.. They give you a very very few set of tricks that let you do complex data-modeling - and you have to be clever and have enough foresight to not block yourself into a hole (or have customer abuse expensive queries).

Then there's Cassandra/Hadoop (HBase). These are petabyte scale databases (technically so is Dynamo/BigTable). They're incredibly efficient at what they do. And they have a lot of plugins to do almost anything you need. I personally love these the best (and RocksDB/LevelDB are like their infant children offspring). You can run these on your laptop (unlike Amazon/Google engines above). But their discipline is very different than all the other's above.

See more
Get Advice from developers at your company using StackShare Enterprise. Sign up for StackShare Enterprise.
Learn More
Pros of OpenTSDB
Pros of SQLite
    Be the first to leave a pro
    • 163
      Lightweight
    • 135
      Portable
    • 122
      Simple
    • 81
      Sql
    • 29
      Preinstalled on iOS and Android
    • 2
      Free
    • 2
      Tcl integration
    • 1
      Portable A database on my USB 'love it'

    Sign up to add or upvote prosMake informed product decisions

    Cons of OpenTSDB
    Cons of SQLite
      Be the first to leave a con
      • 2
        Not for multi-process of multithreaded apps
      • 1
        Needs different binaries for each platform

      Sign up to add or upvote consMake informed product decisions

      - No public GitHub repository available -

      What is OpenTSDB?

      It is a distributed, scalable time series database to store, index & serve metrics collected from computer systems at a large scale. It can store and serve massive amounts of time series data without losing granularity.

      What is SQLite?

      SQLite is an embedded SQL database engine. Unlike most other SQL databases, SQLite does not have a separate server process. SQLite reads and writes directly to ordinary disk files. A complete SQL database with multiple tables, indices, triggers, and views, is contained in a single disk file.

      Need advice about which tool to choose?Ask the StackShare community!

      What companies use OpenTSDB?
      What companies use SQLite?
      See which teams inside your own company are using OpenTSDB or SQLite.
      Sign up for StackShare EnterpriseLearn More

      Sign up to get full access to all the companiesMake informed product decisions

      What tools integrate with OpenTSDB?
      What tools integrate with SQLite?

      Sign up to get full access to all the tool integrationsMake informed product decisions

      Blog Posts

      Jan 26 2022 at 4:34AM

      Pinterest

      Amazon EC2RocksDBOpenTSDB+3
      3
      722
      What are some alternatives to OpenTSDB and SQLite?
      Prometheus
      Prometheus is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true.
      Druid
      Druid is a distributed, column-oriented, real-time analytics data store that is commonly used to power exploratory dashboards in multi-tenant environments. Druid excels as a data warehousing solution for fast aggregate queries on petabyte sized data sets. Druid supports a variety of flexible filters, exact calculations, approximate algorithms, and other useful calculations.
      KairosDB
      KairosDB is a fast distributed scalable time series database written on top of Cassandra.
      InfluxDB
      InfluxDB is a scalable datastore for metrics, events, and real-time analytics. It has a built-in HTTP API so you don't have to write any server side code to get up and running. InfluxDB is designed to be scalable, simple to install and manage, and fast to get data in and out.
      Graphite
      Graphite does two things: 1) Store numeric time-series data and 2) Render graphs of this data on demand
      See all alternatives