InfluxDB

Hi everyone. I'm trying to create my personal syslog monitoring.

1. To get the logs, I have uncertainty to choose the way: 1.1 Use Logstash like a TCP server. 1.2 Implement a Go TCP server.

2. To store and plot data. 2.1 Use Elasticsearch tools. 2.2 Use InfluxDB and Grafana.

I would like to know... Which is a cheaper and scalable solution?

Or even if there is a better way to do it.

Hi all, I am trying to decide on a database for time-series data. The data could be tracking some simple series like statistics over time or could be a nested JSON (multi-level nested). I have been experimenting with InfluxDB for the former case of a simple list of variables over time. The continuous queries are powerful too. But for the latter case, where InfluxDB requires to flatten out a nested JSON before saving it into the database the complexity arises. The nested JSON could be objects or a list of objects and objects under objects in which a complete flattening doesn't leave the data in a state for the queries I'm thinking.

[ 
  { "timestamp": "2021-09-06T12:51:00Z",
    "name": "Name1",
    "books": [
        { "title": "Book1", "page": 100 },
        { "title": "Book2", "page": 280 },
    ]
  },
 { "timestamp": "2021-09-06T12:52:00Z",
   "name": "Name2",
   "books": [
       { "title": "Book1", "page": 320},
       { "title": "Book2", "page": 530 },
       { "title": "Book3", "page": 150 },
   ]
 }
]

Sample query: With a time range, for name xyz, find all the book title for which # of page < 400.

If I flatten it completely, it will result in fields like books_0_title, books_0_page, books_1_title, books_1_page, ... And by losing the nested context it will be hard to return one field (title) where some condition for another field (page) satisfies.

Appreciate any suggestions. Even a piece of generic advice on handling the time-series and choosing the database is welcome!

I am facing a problem of a high latency rate. How can I make it real-time?

My stack is Python (reading some data from serial), InfluxDB (Data Store in DB) React (Front-end show data on a web app with API)

The cycle is Read Data, Store, and then Show Front-end data from API(DB).

We use InfluxDB because we need to store time series data for scan history. It's very simple to use and it's perfect to store value on event (on datetime).

We use InfluxDB as a store for our data that gets fed into Grafana. It's ideal for this as it's a lightweight storage engine that can be modified on the fly by scripts without having to log into the server itself and manage tables. The HTTP API also makes it ideal for integrating with frontend services. InfluxDB

InfluxDB ingests information from various sources (mostly Telegraf instances) into one place for monitoring purposes. InfluxDB

To track time-series of course, utilizing few retention rules and continuous queries to keep time-series data fast and maintanable InfluxDB

Server and application metrics are all stored in InfluxDB for capacity planning and anomaly detection. InfluxDB