Need advice about which tool to choose?Ask the StackShare community!

Grafana

17.4K
13.9K
+ 1
415
Kibana

20.2K
16K
+ 1
261
Add tool

Grafana or Kibana - Help me decide


Context

The observability of applications is an aspect growing in importance every day for software development teams. More observable applications result in improved the productivity of software teams and software organizations as a whole. The benefits of observable applications include:

  • Less time debugging, because more debug information is already available.
  • Resolving issues and incidents faster.
  • Improved awareness of changes in the environment, from operational load to customer behavior.

Two approaches for creating observable applications are monitoring and log analysis.

The monitoring of applications is usually performed by analyzing the changes in discrete data points describing the state of the system at a given moment, called metrics. Metrics are usually submitted directly to the monitoring system by the running instance of an application. That instance can be a database instance, a web server, or any other part of the web service Monitoring systems are generally focused on real-time metrics.

Logs are information about the specific events that took place at a certain moment in time. Log analysis is a post-event inquiry into the log entries, and therefore past events, that a running application produced. Due to the decreasing latency in log processing over the past years, you can now accomplish log analysis in near-real-time.

In this Stackup we look at one tool from each of the two sides: Grafana, a monitoring solution, and Kibana, a log analysis solution that is part of the Elasticsearch, Logstash, and Kibana stack, or ELK.

Use cases

At their core, Grafana and Kibana cover two different use cases and sets of functionality.

Grafana is a monitoring tool, and its functionality is optimized for monitoring tasks and time series data. The data sources it supports are those most commonly used for storing application metrics and Grafana produces alerts in real time.

Kibana, is a data visualization tool. It was created to facilitate log analysis in combination with the popular Elasticsearch and Logstash. The three tools allow you to query and parse relevant information out of the collected logs and display it in different ways.

What's the difference between the two use cases? Grafana focuses on efficiently displaying a defined set of metrics in real time. Kibana focuses on the exploration of available data and the flexibility of extracting metrics from raw log lines.

Comparison

Data sources

Both Grafana and Kibana support Elasticsearch as a data source.

Apart from Elasticsearch, Grafana supports sourcing metrics from:

  • Graphite
  • Prometheus
  • InfluxDB
  • OpenTSDB
  • MySQL, PostgreSQL, Microsoft SQL Server
  • AWS Cloudwatch

Kibana focuses on Elasticsearch and doesn't support any data sources besides Elasticsearch. However, Kibana offers more functionality for the Elasticseach source, like exploring available data and performing a full-text search on the logs.

Querying

With Kibana, you query log lines to produce metrics that you are looking for. For example, if the log lines contain information on HTTP requests:

method=post api=books result=201
method=get api=books result=200
method=get api=bookshelves result=404

If you want to present the amount of successful HTTP queries vs those that didn't return valid results, you do the following:

  • On the machine that produces the example logs above, set up Logstash to process the logs and write them to Elasticsearch.
  • In Kibana, create a time series view that looks for the items that have your desired HTTP statuses.

grafana_vs_kibana_image1.png

A full breakdown of HTTP requests by status, country, OS and other factors in Kibana. Source: elastic.co

Every time the dashboard needs to update, the query runs and produces the most recent counts for the different HTTP statuses.

The main area of the Kibana user interface includes a search box where you can try any Elasticsearch queries, visualize the results, and save the queries that produce the results you are looking for to dashboards.

On dashboards, it is possible to refine the set of data presented by using additional search parameters introduced via a search box (another Elasticsearch query).

Grafana's interface is not optimized for exploring data, but for setting up dashboards once and using them for a long time. Grafana's interface is optimized for time series data, which is the most common visualization type in monitoring systems.

grafana_vs_kibana_image2.jpg

A Grafana dashboard. Source: grafana.org

Like Kibana, Grafana allows you to narrow down the content of the dashboards with variables, a pre-set list of values you can use to filter the output of the visualizations.

Visualizations

Both Grafana and Kibana offer multiple types of data visualizations which you can use on dashboards. While both systems offer visualizations for most common use cases, Kibana goes further and also provides specialized visualizers like maps and tag clouds. Kibana also allows you to embed graphs created with the Vega framework.

You can find the most common visualization types and their availability in both Grafana and Kibana in the table below.

Visualization Grafana Kibana
Time series Yes Yes
Histogram Yes Yes
Heatmap Yes Yes
Single stat Yes Yes
Gauge Yes Yes
Table Yes Yes
Graph No Yes
Map / geospatial data No Yes

Find more details about the supported visualizations in the Grafana and Kibana docs respectively:

Alerting

Grafana has a built-in alerting engine. You can configure alerts for any metric displayed as a time series, and you set via a query like this:

avg() OF query(A, 5m, now) IS BELOW 14

Where A references a metric available in Grafana.

The engine allows handling of special cases like no data available or a failed database connection. If the alert is triggered, Grafana can notify Slack, PagerDuty and other services, or send a generic webhook.

You can find out more about alerting in Grafana in the docs.

Kibana doesn't handle alerts directly but requires you to configure them in Elasticsearch via data watchers. Watchers are functions that run a query periodically and act on the result. You can currently only configure watchers via the API.

Kibana and Elasticsearch currently offer limited documentation on configuring watchers that integrate with third-party services for alerting. Example watchers currently look like this:

https://gist.github.com/skearns64/773dfd64c51d3007baf489be83549e0c

You can find more details about the Elasticsearch Watcher APIs in the documentation.

Conclusion

While monitoring and log analysis solutions contribute to the observability of applications, the tools from the two camps solve different problems and are complementary.

Collecting metrics allows the teams responsible for applications to gain visibility into the current state of a system in real time. The application needs to submit these metrics, and changing the exact metrics submitted generally requires application changes. Collecting metrics is not always possible for legacy or closed-source applications where the team operating the system doesn't have access to the code. But if you can build metrics collection into your application, then collecting and visualizing metrics is where Grafana excels.

Log analysis makes it possible to analyze events produced by the application, which is sometimes the only way to gain insight into the state of a closed system that does not produce relevant metrics. For applications that do produce metrics, log analysis can allow operators to find new trends in the system behavior and iterate on the metrics quickly without application changes. When used as part of the ELK stack, this is where Kibana excels.

Grafana vs Kibana: What are the differences?

Grafana is an open-source analytics and monitoring platform, while Kibana is a data visualization and exploration tool. Let's explore the key differences between them.

  1. Data Source Compatibility: Grafana supports a wide range of data sources including popular databases, cloud platforms, and monitoring tools, making it versatile for data visualization. On the other hand, Kibana is primarily designed to work with Elasticsearch, providing advanced data analysis and visualization capabilities specific to this platform.

  2. Purpose and Focus: Grafana focuses on providing a comprehensive platform for creating visually appealing dashboards and monitoring various data sources. It excels in time series data analysis and dashboard customization. In contrast, Kibana is more focused on log and event data analysis, offering powerful search capabilities, log aggregation, and anomaly detection.

  3. Plugin and Extension Ecosystem: Grafana has a vibrant community-driven ecosystem with extensive plugin and extension support. This allows users to extend the functionality, integrate with additional data sources, and customize their dashboards extensively. On the other hand, Kibana has a more limited plugin ecosystem, as it primarily relies on Elasticsearch's functionalities for data analysis.

  4. Alerting and Notification System: Grafana provides a built-in alerting and notification system, which allows users to configure and receive notifications based on specified thresholds or conditions. Kibana, on the other hand, lacks a dedicated built-in alerting mechanism, and users often rely on external tools or scripts to achieve similar functionality.

  5. User Interface and Visualization Capabilities: Grafana offers a user-friendly interface with a wide range of visualization options, including graphs, charts, tables, and maps. It provides a drag-and-drop editor for creating and customizing visualizations easily. Kibana also offers a visually appealing interface but is more focused on log analysis and visualizations specific to Elasticsearch, such as aggregations, time series visualizations, and geospatial analysis.

  6. Community and User Support: Grafana has a large and active community of users, making it easier to find resources, tutorials, and community-driven plugins. Kibana, being part of the Elastic Stack, also has a strong community but may have comparatively less diverse resources available outside the Elastic ecosystem. Additionally, Grafana has been widely adopted by various companies and organizations, further bolstering its community and user support.

In summary, Grafana excels in its data source compatibility, dashboard customization, and vibrant plugin ecosystem, making it a versatile platform for data visualization. On the other hand, Kibana focuses on log and event data analysis, providing powerful search capabilities but with a more specific focus on Elasticsearch.

Advice on Grafana and Kibana
Susmita Meher
Senior SRE at African Bank · | 4 upvotes · 780.2K views
Needs advice
on
GrafanaGrafanaGraphiteGraphite
and
PrometheusPrometheus

Looking for a tool which can be used for mainly dashboard purposes, but here are the main requirements:

  • Must be able to get custom data from AS400,
  • Able to display automation test results,
  • System monitoring / Nginx API,
  • Able to get data from 3rd parties DB.

Grafana is almost solving all the problems, except AS400 and no database to get automation test results.

See more
Replies (1)
Sakti Behera
Technical Specialist, Software Engineering at AT&T · | 3 upvotes · 565.5K views
Recommends
on
GrafanaGrafanaPrometheusPrometheus

You can look out for Prometheus Instrumentation (https://prometheus.io/docs/practices/instrumentation/) Client Library available in various languages https://prometheus.io/docs/instrumenting/clientlibs/ to create the custom metric you need for AS4000 and then Grafana can query the newly instrumented metric to show on the dashboard.

See more
Mat Jovanovic
Head of Cloud at Mats Cloud · | 3 upvotes · 709.6K views
Needs advice
on
DatadogDatadogGrafanaGrafana
and
PrometheusPrometheus

We're looking for a Monitoring and Logging tool. It has to support AWS (mostly 100% serverless, Lambdas, SNS, SQS, API GW, CloudFront, Autora, etc.), as well as Azure and GCP (for now mostly used as pure IaaS, with a lot of cognitive services, and mostly managed DB). Hopefully, something not as expensive as Datadog or New relic, as our SRE team could support the tool inhouse. At the moment, we primarily use CloudWatch for AWS and Pandora for most on-prem.

See more
Replies (2)
Lucas Rincon
Recommends
on
InstanaInstana

this is quite affordable and provides what you seem to be looking for. you can see a whole thing about the APM space here https://www.apmexperts.com/observability/ranking-the-observability-offerings/

See more
Recommends
on
DatadogDatadog

I worked with Datadog at least one year and my position is that commercial tools like Datadog are the best option to consolidate and analyze your metrics. Obviously, if you can't pay the tool, the best free options are the mix of Prometheus with their Alert Manager and Grafana to visualize (that are complementary not substitutable). But I think that no use a good tool it's finally more expensive that use a not really good implementation of free tools and you will pay also to maintain its.

See more
Needs advice
on
GrafanaGrafana
and
KibanaKibana

From a StackShare Community member: “We need better analytics & insights into our Elasticsearch cluster. Grafana, which ships with advanced support for Elasticsearch, looks great but isn’t officially supported/endorsed by Elastic. Kibana, on the other hand, is made and supported by Elastic. I’m wondering what people suggest in this situation."

See more
Replies (7)
Recommends
on
GrafanaGrafana
at

For our Predictive Analytics platform, we have used both Grafana and Kibana

Kibana has predictions and ML algorithms support, so if you need them, you may be better off with Kibana . The multi-variate analysis features it provide are very unique (not available in Grafana).

For everything else, definitely Grafana . Especially the number of supported data sources, and plugins clearly makes Grafana a winner (in just visualization and reporting sense). Creating your own plugin is also very easy. The top pros of Grafana (which it does better than Kibana ) are:

  • Creating and organizing visualization panels
  • Templating the panels on dashboards for repetetive tasks
  • Realtime monitoring, filtering of charts based on conditions and variables
  • Export / Import in JSON format (that allows you to version and save your dashboard as part of git)
See more
Recommends
on
KibanaKibana

I use both Kibana and Grafana on my workplace: Kibana for logging and Grafana for monitoring. Since you already work with Elasticsearch, I think Kibana is the safest choice in terms of ease of use and variety of messages it can manage, while Grafana has still (in my opinion) a strong link to metrics

See more
Bram Verdonck
Recommends
on
GrafanaGrafana
at

After looking for a way to monitor or at least get a better overview of our infrastructure, we found out that Grafana (which I previously only used in ELK stacks) has a plugin available to fully integrate with Amazon CloudWatch . Which makes it way better for our use-case than the offer of the different competitors (most of them are even paid). There is also a CloudFlare plugin available, the platform we use to serve our DNS requests. Although we are a big fan of https://smashing.github.io/ (previously dashing), for now we are starting with Grafana .

See more
Recommends
on
KibanaKibana

I use Kibana because it ships with the ELK stack. I don't find it as powerful as Splunk however it is light years above grepping through log files. We previously used Grafana but found it to be annoying to maintain a separate tool outside of the ELK stack. We were able to get everything we needed from Kibana.

See more
Recommends
on
KibanaKibana

Kibana should be sufficient in this architecture for decent analytics, if stronger metrics is needed then combine with Grafana. Datadog also offers nice overview but there's no need for it in this case unless you need more monitoring and alerting (and more technicalities).

See more
Recommends
on
GrafanaGrafana

I use Grafana because it is without a doubt the best way to visualize metrics

See more
Povilas Brilius
PHP Web Developer at GroundIn Software · | 0 upvotes · 590.9K views
Recommends
on
KibanaKibana
at

@Kibana, of course, because @Grafana looks like amateur sort of solution, crammed with query builder grouping aggregates, but in essence, as recommended by CERN - KIbana is the corporate (startup vectored) decision.

Furthermore, @Kibana comes with complexity adhering ELK stack, whereas @InfluxDB + @Grafana & co. recently have become sophisticated development conglomerate instead of advancing towards a understandable installation step by step inheritance.

See more
Decisions about Grafana and Kibana
Matt Menzenski
Senior Software Engineering Manager at PayIt · | 15 upvotes · 984.1K views

Grafana and Prometheus together, running on Kubernetes , is a powerful combination. These tools are cloud-native and offer a large community and easy integrations. At PayIt we're using exporting Java application metrics using a Dropwizard metrics exporter, and our Node.js services now use the prom-client npm library to serve metrics.

See more
Leonardo Henrique da Paixão
Junior QA Tester at SolarMarket · | 2 upvotes · 173.8K views

I learned a lot from Grafana, especially the issue of data monitoring, as it is easy to use, I learned how to create quick and simple dashboards. InfluxDB, I didn't know any other types of DBMS, I only knew about relational DBMS or not, but the difference was the scalability of both, but with influxDB, I knew how a time series DBMS works and finally, Telegraf, which is from the same company as InfluxDB, as I used the Windows Operating System, Telegraf tools was the first in the industry, in addition, it has complete documentation, facilitating its use, I learned a lot about connections, without having to make scripts to collect the data.

See more
Leonardo Henrique da Paixão
Junior QA Tester at SolarMarket · | 15 upvotes · 351.7K views

The objective of this work was to develop a system to monitor the materials of a production line using IoT technology. Currently, the process of monitoring and replacing parts depends on manual services. For this, load cells, microcontroller, Broker MQTT, Telegraf, InfluxDB, and Grafana were used. It was implemented in a workflow that had the function of collecting sensor data, storing it in a database, and visualizing it in the form of weight and quantity. With these developed solutions, he hopes to contribute to the logistics area, in the replacement and control of materials.

See more
Get Advice from developers at your company using StackShare Enterprise. Sign up for StackShare Enterprise.
Learn More
What are some alternatives to Grafana and Kibana?
Datadog
Datadog is the leading service for cloud-scale monitoring. It is used by IT, operations, and development teams who build and operate applications that run on dynamic or hybrid cloud infrastructure. Start monitoring in minutes with Datadog!
Prometheus
Prometheus is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true.
Graphite
Graphite does two things: 1) Store numeric time-series data and 2) Render graphs of this data on demand
Splunk
It provides the leading platform for Operational Intelligence. Customers use it to search, monitor, analyze and visualize machine data.
Netdata
Netdata collects metrics per second & presents them in low-latency dashboards. It's designed to run on all of your physical & virtual servers, cloud deployments, Kubernetes clusters & edge/IoT devices, to monitor systems, containers & apps
See all alternatives