Grafana
Grafana

2.4K
299
470
Kibana
Kibana

3.7K
222
453
Add tool

Grafana or Kibana - Help me decide


Context

The observability of applications is an aspect growing in importance every day for software development teams. More observable applications result in improved the productivity of software teams and software organizations as a whole. The benefits of observable applications include:

  • Less time debugging, because more debug information is already available.
  • Resolving issues and incidents faster.
  • Improved awareness of changes in the environment, from operational load to customer behavior.

Two approaches for creating observable applications are monitoring and log analysis.

The monitoring of applications is usually performed by analyzing the changes in discrete data points describing the state of the system at a given moment, called metrics. Metrics are usually submitted directly to the monitoring system by the running instance of an application. That instance can be a database instance, a web server, or any other part of the web service Monitoring systems are generally focused on real-time metrics.

Logs are information about the specific events that took place at a certain moment in time. Log analysis is a post-event inquiry into the log entries, and therefore past events, that a running application produced. Due to the decreasing latency in log processing over the past years, you can now accomplish log analysis in near-real-time.

In this Stackup we look at one tool from each of the two sides: Grafana, a monitoring solution, and Kibana, a log analysis solution that is part of the Elasticsearch, Logstash, and Kibana stack, or ELK.

Use cases

At their core, Grafana and Kibana cover two different use cases and sets of functionality.

Grafana is a monitoring tool, and its functionality is optimized for monitoring tasks and time series data. The data sources it supports are those most commonly used for storing application metrics and Grafana produces alerts in real time.

Kibana, is a data visualization tool. It was created to facilitate log analysis in combination with the popular Elasticsearch and Logstash. The three tools allow you to query and parse relevant information out of the collected logs and display it in different ways.

What's the difference between the two use cases? Grafana focuses on efficiently displaying a defined set of metrics in real time. Kibana focuses on the exploration of available data and the flexibility of extracting metrics from raw log lines.

Comparison

Data sources

Both Grafana and Kibana support Elasticsearch as a data source.

Apart from Elasticsearch, Grafana supports sourcing metrics from:

  • Graphite
  • Prometheus
  • InfluxDB
  • OpenTSDB
  • MySQL, PostgreSQL, Microsoft SQL Server
  • AWS Cloudwatch

Kibana focuses on Elasticsearch and doesn't support any data sources besides Elasticsearch. However, Kibana offers more functionality for the Elasticseach source, like exploring available data and performing a full-text search on the logs.

Querying

With Kibana, you query log lines to produce metrics that you are looking for. For example, if the log lines contain information on HTTP requests:

method=post api=books result=201
method=get api=books result=200
method=get api=bookshelves result=404

If you want to present the amount of successful HTTP queries vs those that didn't return valid results, you do the following:

  • On the machine that produces the example logs above, set up Logstash to process the logs and write them to Elasticsearch.
  • In Kibana, create a time series view that looks for the items that have your desired HTTP statuses.

alt_text

A full breakdown of HTTP requests by status, country, OS and other factors in Kibana. Source: elastic.co

Every time the dashboard needs to update, the query runs and produces the most recent counts for the different HTTP statuses.

The main area of the Kibana user interface includes a search box where you can try any Elasticsearch queries, visualize the results, and save the queries that produce the results you are looking for to dashboards.

On dashboards, it is possible to refine the set of data presented by using additional search parameters introduced via a search box (another Elasticsearch query).

Grafana's interface is not optimized for exploring data, but for setting up dashboards once and using them for a long time. Grafana's interface is optimized for time series data, which is the most common visualization type in monitoring systems.

alt_text

A Grafana dashboard. Source: grafana.org

Like Kibana, Grafana allows you to narrow down the content of the dashboards with variables, a pre-set list of values you can use to filter the output of the visualizations.

Visualizations

Both Grafana and Kibana offer multiple types of data visualizations which you can use on dashboards. While both systems offer visualizations for most common use cases, Kibana goes further and also provides specialized visualizers like maps and tag clouds. Kibana also allows you to embed graphs created with the Vega framework.

You can find the most common visualization types and their availability in both Grafana and Kibana in the table below.

Visualization Grafana Kibana
Time series Yes Yes
Histogram Yes Yes
Heatmap Yes Yes
Single stat Yes Yes
Gauge Yes Yes
Table Yes Yes
Graph No Yes
Map / geospatial data No Yes

Find more details about the supported visualizations in the Grafana and Kibana docs respectively:

Alerting

Grafana has a built-in alerting engine. You can configure alerts for any metric displayed as a time series, and you set via a query like this:

avg() OF query(A, 5m, now) IS BELOW 14

Where A references a metric available in Grafana.

The engine allows handling of special cases like no data available or a failed database connection. If the alert is triggered, Grafana can notify Slack, PagerDuty and other services, or send a generic webhook.

You can find out more about alerting in Grafana in the docs.

Kibana doesn't handle alerts directly but requires you to configure them in Elasticsearch via data watchers. Watchers are functions that run a query periodically and act on the result. You can currently only configure watchers via the API.

Kibana and Elasticsearch currently offer limited documentation on configuring watchers that integrate with third-party services for alerting. Example watchers currently look like this:

https://gist.github.com/skearns64/773dfd64c51d3007baf489be83549e0c

You can find more details about the Elasticsearch Watcher APIs in the documentation.

Conclusion

While monitoring and log analysis solutions contribute to the observability of applications, the tools from the two camps solve different problems and are complementary.

Collecting metrics allows the teams responsible for applications to gain visibility into the current state of a system in real time. The application needs to submit these metrics, and changing the exact metrics submitted generally requires application changes. Collecting metrics is not always possible for legacy or closed-source applications where the team operating the system doesn't have access to the code. But if you can build metrics collection into your application, then collecting and visualizing metrics is where Grafana excels.

Log analysis makes it possible to analyze events produced by the application, which is sometimes the only way to gain insight into the state of a closed system that does not produce relevant metrics. For applications that do produce metrics, log analysis can allow operators to find new trends in the system behavior and iterate on the metrics quickly without application changes. When used as part of the ELK stack, this is where Kibana excels.

What is Grafana?

Grafana is a general purpose dashboard and graph composer. It's focused on providing rich ways to visualize time series metrics, mainly though graphs but supports other ways to visualize data through a pluggable panel architecture. It currently has rich support for for Graphite, InfluxDB and OpenTSDB. But supports other data sources via plugins.

What is Kibana?

Kibana is an open source (Apache Licensed), browser based analytics and search dashboard for Elasticsearch. Kibana is a snap to setup and start using. Kibana strives to be easy to get started with, while also being flexible and powerful, just like Elasticsearch.

Want advice about which of these to choose?Ask the StackShare community!

Why do developers choose Grafana?
Why do developers choose Kibana?
What are the cons of using Grafana?
What are the cons of using Kibana?
Be the first to leave a con
What companies use Grafana?
What companies use Kibana?
What are some alternatives to Grafana and Kibana?
Loggly
The world's most popular cloud-based log management service delivers application intelligence.
Graylog
Centralize and aggregate all your log files for 100% visibility. Use our powerful query language to search through terabytes of log data to discover and analyze important information.
Splunk
Splunk Inc. provides the leading platform for Operational Intelligence. Customers use Splunk to search, monitor, analyze and visualize machine data.
Prometheus
Prometheus is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true.
Tableau
Tableau can help anyone see and understand their data. Connect to almost any database, drag and drop to create visualizations, and share with a click.
See all alternatives
What tools integrate with Grafana?
What tools integrate with Kibana?
Decisions about Grafana and Kibana
No stack decisions found
Interest over time
Reviews of Grafana and Kibana
Review ofGrafanaGrafana

analyze heap dump and many logging or traces

How developers use Grafana and Kibana
Avatar of ShadowICT
ShadowICT uses GrafanaGrafana

We use Grafana to view live stats relating to our servers such as memory and CPU usage. We also use Grafana to monitor our gaming servers for data such as latency and player counts. This allows us to generate effective analytics and see when problems arise.

Avatar of Andrew Gatenby
Andrew Gatenby uses GrafanaGrafana

Everyone likes graphs, right?! This isn't a tool we actively use right now, but paired with Prometheus we want to use it to have visual monitors on things like API cluster health, status, queue stats, DB/redis query and cache stats etc.

Avatar of Scrayos UG (haftungsbeschränkt)
Scrayos UG (haftungsbeschränkt) uses GrafanaGrafana

Grafana is used in combination with Prometheus to display the gathered stats and to monitor our physical servers aswell as their virtual applications. We also use Grafana to get notifications about irregularities.

Avatar of sapslaj
sapslaj uses GrafanaGrafana

Grafana takes the data from InfluxDB and presents it in a nice flexible format. Bonus points for built-in alerts and playlists (cycles through different dashboards automatically)

Avatar of Bùi Thanh
Bùi Thanh uses GrafanaGrafana
  • Graph report with many panels and Dashboard.
  • Easy to deploy, and view performance of system.
  • Intergrating with many datasource: Prometheus, CloudWatch
  • Alerts
Avatar of Clarabridge Engage
Clarabridge Engage uses KibanaKibana

Used for graphing internal logging data; including metrics related to how fast we serve pages and execute MySQL/ElasticSearch queries.

Avatar of Wirkn Inc.
Wirkn Inc. uses KibanaKibana

Our Kibana instances uses our ElasticSearch search data to help answer any complicated questions we have about our data.

Avatar of Hevelop
Hevelop uses KibanaKibana

Kibana is our tools to query data in Elasticsearch clusters set up as catalog search engine.

Avatar of Diogo Silva
Diogo Silva uses KibanaKibana

Perfect for exploring and visualizing the data available at ElasticSearch

Avatar of Tongliang Liu
Tongliang Liu uses KibanaKibana

Log visualization. Wish it could add built-in alert functionality.

How much does Grafana cost?
How much does Kibana cost?
Pricing unavailable