What is Riemann and what are its top alternatives?
Riemann is an open-source event stream processing system designed for monitoring and data processing in real-time. It can handle high volumes of events and offers powerful stream processing capabilities to perform analysis, aggregation, and alerting on data streams. However, setting up and configuring Riemann can be complex and requires knowledge of the Clojure programming language.
Prometheus: Prometheus is a popular open-source monitoring and alerting toolkit built at SoundCloud. It provides a flexible query language, powerful data visualization tools, and seamless integration with Grafana for dashboards. Prometheus offers a scalable and reliable solution for monitoring various systems and applications.
InfluxDB: InfluxDB is a time-series database optimized for high-performance metrics and events storage. It offers a simple SQL-like query language, high availability, and horizontal scalability. InfluxDB is commonly used for storing and querying monitoring data from various sources.
Grafana: Grafana is an open-source analytics and monitoring platform that allows you to query, visualize, and alert on metrics and logs from multiple sources. It supports a wide range of data sources, including Prometheus, InfluxDB, and Elasticsearch, making it a versatile tool for creating comprehensive dashboards and alerts.
Elasticsearch: Elasticsearch is a distributed search engine commonly used for storing and searching log data. It offers powerful full-text search capabilities, real-time indexing, and high availability. Elasticsearch is often paired with Kibana for data visualization and monitoring.
Nagios: Nagios is a popular open-source monitoring system known for its flexibility and extensibility. It can monitor servers, networks, and applications using a variety of plugins and provides alerting capabilities for detecting and resolving issues proactively.
Zabbix: Zabbix is an enterprise-grade open-source monitoring solution capable of monitoring various metrics, performance counters, and network devices. It offers a web-based interface, customizable alerting, and reporting features for tracking the health of your infrastructure.
SignalFx: SignalFx is a cloud monitoring and observability platform that provides real-time analytics, visualization, and alerting for cloud-native applications. It offers automated detection of anomalies, proactive alerting, and integration with popular cloud services.
Dynatrace: Dynatrace is an AI-powered monitoring and observability platform designed for cloud-native environments. It offers automatic discovery and tracking of dependencies, intelligent root cause analysis, and precise alerting to help teams optimize performance and availability.
Splunk: Splunk is a data analytics platform that specializes in searching, monitoring, and analyzing machine-generated data. It offers real-time insights, visualization tools, and machine learning capabilities for investigating and troubleshooting issues across your infrastructure.
OpenTSDB: OpenTSDB is a distributed, scalable time-series database built on top of HBase. It is optimized for storing and querying large amounts of time-series data, making it suitable for monitoring and analyzing metrics from various sources. OpenTSDB can be integrated with Grafana for visualization and analytics.
Top Alternatives to Riemann
- Prometheus
Prometheus is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true. ...
- Kafka
Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design. ...
- Grafana
Grafana is a general purpose dashboard and graph composer. It's focused on providing rich ways to visualize time series metrics, mainly though graphs but supports other ways to visualize data through a pluggable panel architecture. It currently has rich support for for Graphite, InfluxDB and OpenTSDB. But supports other data sources via plugins. ...
- Kapacitor
It is a native data processing engine for InfluxDB 1.x and is an integrated component in the InfluxDB 2.0 platform. It can process both stream and batch data from InfluxDB, acting on this data in real-time via its programming language TICKscript. ...
- Nagios
Nagios is a host/service/network monitoring program written in C and released under the GNU General Public License. ...
- Beats
Beats is the platform for single-purpose data shippers. They send data from hundreds or thousands of machines and systems to Logstash or Elasticsearch. ...
- Wireshark
It is the world’s foremost and widely-used network protocol analyzer. It lets you see what’s happening on your network at a microscopic level and is the de facto standard across many commercial and non-profit enterprises, government agencies, and educational institutions. ...
- PRTG
It can monitor and classify system conditions like bandwidth usage or uptime and collect statistics from miscellaneous hosts as switches, routers, servers and other devices and applications. ...
Riemann alternatives & related posts
Prometheus
- Powerful easy to use monitoring47
- Flexible query language38
- Dimensional data model32
- Alerts27
- Active and responsive community23
- Extensive integrations22
- Easy to setup19
- Beautiful Model and Query language12
- Easy to extend7
- Nice6
- Written in Go3
- Good for experimentation2
- Easy for monitoring1
- Just for metrics12
- Bad UI6
- Needs monitoring to access metrics endpoints6
- Not easy to configure and use4
- Supports only active agents3
- Written in Go2
- TLS is quite difficult to understand2
- Requires multiple applications and tools2
- Single point of failure1
related Prometheus posts
Why we spent several years building an open source, large-scale metrics alerting system, M3, built for Prometheus:
By late 2014, all services, infrastructure, and servers at Uber emitted metrics to a Graphite stack that stored them using the Whisper file format in a sharded Carbon cluster. We used Grafana for dashboarding and Nagios for alerting, issuing Graphite threshold checks via source-controlled scripts. While this worked for a while, expanding the Carbon cluster required a manual resharding process and, due to lack of replication, any single node’s disk failure caused permanent loss of its associated metrics. In short, this solution was not able to meet our needs as the company continued to grow.
To ensure the scalability of Uber’s metrics backend, we decided to build out a system that provided fault tolerant metrics ingestion, storage, and querying as a managed platform...
(GitHub : https://github.com/m3db/m3)
Grafana and Prometheus together, running on Kubernetes , is a powerful combination. These tools are cloud-native and offer a large community and easy integrations. At PayIt we're using exporting Java application metrics using a Dropwizard metrics exporter, and our Node.js services now use the prom-client npm library to serve metrics.
- High-throughput126
- Distributed119
- Scalable92
- High-Performance86
- Durable66
- Publish-Subscribe38
- Simple-to-use19
- Open source18
- Written in Scala and java. Runs on JVM12
- Message broker + Streaming system9
- KSQL4
- Avro schema integration4
- Robust4
- Suport Multiple clients3
- Extremely good parallelism constructs2
- Partioned, replayable log2
- Simple publisher / multi-subscriber model1
- Fun1
- Flexible1
- Non-Java clients are second-class citizens32
- Needs Zookeeper29
- Operational difficulties9
- Terrible Packaging5
related Kafka posts
The algorithms and data infrastructure at Stitch Fix is housed in #AWS. Data acquisition is split between events flowing through Kafka, and periodic snapshots of PostgreSQL DBs. We store data in an Amazon S3 based data warehouse. Apache Spark on Yarn is our tool of choice for data movement and #ETL. Because our storage layer (s3) is decoupled from our processing layer, we are able to scale our compute environment very elastically. We have several semi-permanent, autoscaling Yarn clusters running to serve our data processing needs. While the bulk of our compute infrastructure is dedicated to algorithmic processing, we also implemented Presto for adhoc queries and dashboards.
Beyond data movement and ETL, most #ML centric jobs (e.g. model training and execution) run in a similarly elastic environment as containers running Python and R code on Amazon EC2 Container Service clusters. The execution of batch jobs on top of ECS is managed by Flotilla, a service we built in house and open sourced (see https://github.com/stitchfix/flotilla-os).
At Stitch Fix, algorithmic integrations are pervasive across the business. We have dozens of data products actively integrated systems. That requires serving layer that is robust, agile, flexible, and allows for self-service. Models produced on Flotilla are packaged for deployment in production using Khan, another framework we've developed internally. Khan provides our data scientists the ability to quickly productionize those models they've developed with open source frameworks in Python 3 (e.g. PyTorch, sklearn), by automatically packaging them as Docker containers and deploying to Amazon ECS. This provides our data scientist a one-click method of getting from their algorithms to production. We then integrate those deployments into a service mesh, which allows us to A/B test various implementations in our product.
For more info:
- Our Algorithms Tour: https://algorithms-tour.stitchfix.com/
- Our blog: https://multithreaded.stitchfix.com/blog/
- Careers: https://multithreaded.stitchfix.com/careers/
#DataScience #DataStack #Data
As we've evolved or added additional infrastructure to our stack, we've biased towards managed services. Most new backing stores are Amazon RDS instances now. We do use self-managed PostgreSQL with TimescaleDB for time-series data—this is made HA with the use of Patroni and Consul.
We also use managed Amazon ElastiCache instances instead of spinning up Amazon EC2 instances to run Redis workloads, as well as shifting to Amazon Kinesis instead of Kafka.
- Beautiful89
- Graphs are interactive68
- Free57
- Easy56
- Nicer than the Graphite web interface34
- Many integrations26
- Can build dashboards18
- Easy to specify time window10
- Can collaborate on dashboards10
- Dashboards contain number tiles9
- Open Source5
- Integration with InfluxDB5
- Click and drag to zoom in5
- Authentification and users management4
- Threshold limits in graphs4
- Alerts3
- It is open to cloud watch and many database3
- Simple and native support to Prometheus3
- Great community support2
- You can use this for development to check memcache2
- You can visualize real time data to put alerts2
- Grapsh as code0
- Plugin visualizationa0
- No interactive query builder1
related Grafana posts
Why we spent several years building an open source, large-scale metrics alerting system, M3, built for Prometheus:
By late 2014, all services, infrastructure, and servers at Uber emitted metrics to a Graphite stack that stored them using the Whisper file format in a sharded Carbon cluster. We used Grafana for dashboarding and Nagios for alerting, issuing Graphite threshold checks via source-controlled scripts. While this worked for a while, expanding the Carbon cluster required a manual resharding process and, due to lack of replication, any single node’s disk failure caused permanent loss of its associated metrics. In short, this solution was not able to meet our needs as the company continued to grow.
To ensure the scalability of Uber’s metrics backend, we decided to build out a system that provided fault tolerant metrics ingestion, storage, and querying as a managed platform...
(GitHub : https://github.com/m3db/m3)
Grafana and Prometheus together, running on Kubernetes , is a powerful combination. These tools are cloud-native and offer a large community and easy integrations. At PayIt we're using exporting Java application metrics using a Dropwizard metrics exporter, and our Node.js services now use the prom-client npm library to serve metrics.
related Kapacitor posts
Nagios
- It just works53
- The standard28
- Customizable12
- The Most flexible monitoring system8
- Huge stack of free checks/plugins to choose from1
related Nagios posts
Why we spent several years building an open source, large-scale metrics alerting system, M3, built for Prometheus:
By late 2014, all services, infrastructure, and servers at Uber emitted metrics to a Graphite stack that stored them using the Whisper file format in a sharded Carbon cluster. We used Grafana for dashboarding and Nagios for alerting, issuing Graphite threshold checks via source-controlled scripts. While this worked for a while, expanding the Carbon cluster required a manual resharding process and, due to lack of replication, any single node’s disk failure caused permanent loss of its associated metrics. In short, this solution was not able to meet our needs as the company continued to grow.
To ensure the scalability of Uber’s metrics backend, we decided to build out a system that provided fault tolerant metrics ingestion, storage, and querying as a managed platform...
(GitHub : https://github.com/m3db/m3)
I am new to DevOps and looking for training in DevOps. Some institutes are offering Nagios while some Prometheus in their syllabus. Please suggest which one is being used in the industry and which one should I learn.
related Beats posts
related Wireshark posts
- Poor search capabilities1
- Graphs are static1
- Running on windows1