Need advice about which tool to choose?Ask the StackShare community!

Loki

500
312
+ 1
17
Prometheus

4.5K
3.7K
+ 1
239
Add tool

Loki vs Prometheus: What are the differences?

Introduction

Loki and Prometheus are both open-source logging and monitoring tools used for observability in modern software systems. While they have some similarities, there are key differences that set them apart in terms of architecture, data model, and use cases.

  1. Data Model: Prometheus stores time-series data and provides a multi-dimensional data model, where metrics are identified by their name and a set of key-value pairs called labels. On the other hand, Loki is designed specifically for log data, storing logs as streams of events with labels attached to each log line.

  2. Querying Capability: Prometheus offers a powerful querying language called PromQL, which allows users to retrieve and analyze time-series data efficiently. It supports aggregations, mathematical operations, and functions tailored for time-series analysis. Loki, however, provides a log-specific query language called LogQL that enables users to search, filter, and aggregate logs based on labels and values.

  3. Storage Architecture: Prometheus follows a pull-based model, where it scrapes metrics from instrumented applications at regular intervals. It stores the data locally in a time-series database (TSDB). In contrast, Loki employs a push-based model, where applications send logs directly to Loki. Logs are then indexed and stored in a distributed storage backend, such as object storage or a distributed filesystem.

  4. Retention and Scalability: Prometheus has a default retention period for metrics, typically a few weeks, depending on the disk space available. It supports horizontal scalability through federation and sharding. In contrast, Loki is designed for long-term log retention, usually months or years, and supports horizontal scalability through chunking and replication across multiple instances.

  5. Alerting and Monitoring: Prometheus has built-in alerting capabilities, allowing users to define alerting rules based on metrics and send alerts via various channels. It also provides a powerful visual dashboard, Grafana, for monitoring and visualization. Loki, on the other hand, does not have native alerting capabilities and relies on integrating with other tools like Promtail, Grafana alerts, or external alerting systems. It provides less real-time monitoring and focuses more on log analysis and troubleshooting.

  6. Use Cases: Prometheus is well-suited for monitoring the performance, availability, and health of applications and infrastructure components using metrics. It excels in providing real-time insights and alerting based on predefined thresholds. Loki, on the other hand, is more suitable for log analysis, troubleshooting, and debugging of distributed systems. It helps in investigating and correlating logs across multiple sources to understand the behavior of applications and detect anomalies.

In summary, Prometheus is primarily focused on metrics-based monitoring and alerting, while Loki is tailored for log analysis and troubleshooting in distributed systems.

Advice on Loki and Prometheus
Susmita Meher
Senior SRE at African Bank · | 4 upvotes · 781.1K views
Needs advice
on
GrafanaGrafanaGraphiteGraphite
and
PrometheusPrometheus

Looking for a tool which can be used for mainly dashboard purposes, but here are the main requirements:

  • Must be able to get custom data from AS400,
  • Able to display automation test results,
  • System monitoring / Nginx API,
  • Able to get data from 3rd parties DB.

Grafana is almost solving all the problems, except AS400 and no database to get automation test results.

See more
Replies (1)
Sakti Behera
Technical Specialist, Software Engineering at AT&T · | 3 upvotes · 566.4K views
Recommends
on
GrafanaGrafanaPrometheusPrometheus

You can look out for Prometheus Instrumentation (https://prometheus.io/docs/practices/instrumentation/) Client Library available in various languages https://prometheus.io/docs/instrumenting/clientlibs/ to create the custom metric you need for AS4000 and then Grafana can query the newly instrumented metric to show on the dashboard.

See more
Sunil Chaudhari
Needs advice
on
MetricbeatMetricbeat
and
PrometheusPrometheus

Hi, We have a situation, where we are using Prometheus to get system metrics from PCF (Pivotal Cloud Foundry) platform. We send that as time-series data to Cortex via a Prometheus server and built a dashboard using Grafana. There is another pipeline where we need to read metrics from a Linux server using Metricbeat, CPU, memory, and Disk. That will be sent to Elasticsearch and Grafana will pull and show the data in a dashboard.

Is it OK to use Metricbeat for Linux server or can we use Prometheus?

What is the difference in system metrics sent by Metricbeat and Prometheus node exporters?

Regards, Sunil.

See more
Replies (2)
Matthew Rothstein
Recommends
on
PrometheusPrometheus

If you're already using Prometheus for your system metrics, then it seems like standing up Elasticsearch just for Linux host monitoring is excessive. The node_exporter is probably sufficient if you'e looking for standard system metrics.

Another thing to consider is that Metricbeat / ELK use a push model for metrics delivery, whereas Prometheus pulls metrics from each node it is monitoring. Depending on how you manage your network security, opting for one solution over two may make things simpler.

See more
Recommends
on
InstanaInstana

Hi Sunil! Unfortunately, I don´t have much experience with Metricbeat so I can´t advise on the diffs with Prometheus...for Linux server, I encourage you to use Prometheus node exporter and for PCF, I would recommend using the instana tile (https://www.instana.com/supported-technologies/pivotal-cloud-foundry/). Let me know if you have further questions! Regards Jose

See more
Mat Jovanovic
Head of Cloud at Mats Cloud · | 3 upvotes · 710.5K views
Needs advice
on
DatadogDatadogGrafanaGrafana
and
PrometheusPrometheus

We're looking for a Monitoring and Logging tool. It has to support AWS (mostly 100% serverless, Lambdas, SNS, SQS, API GW, CloudFront, Autora, etc.), as well as Azure and GCP (for now mostly used as pure IaaS, with a lot of cognitive services, and mostly managed DB). Hopefully, something not as expensive as Datadog or New relic, as our SRE team could support the tool inhouse. At the moment, we primarily use CloudWatch for AWS and Pandora for most on-prem.

See more
Replies (2)
Lucas Rincon
Recommends
on
InstanaInstana

this is quite affordable and provides what you seem to be looking for. you can see a whole thing about the APM space here https://www.apmexperts.com/observability/ranking-the-observability-offerings/

See more
Recommends
on
DatadogDatadog

I worked with Datadog at least one year and my position is that commercial tools like Datadog are the best option to consolidate and analyze your metrics. Obviously, if you can't pay the tool, the best free options are the mix of Prometheus with their Alert Manager and Grafana to visualize (that are complementary not substitutable). But I think that no use a good tool it's finally more expensive that use a not really good implementation of free tools and you will pay also to maintain its.

See more
Get Advice from developers at your company using StackShare Enterprise. Sign up for StackShare Enterprise.
Learn More
Pros of Loki
Pros of Prometheus
  • 5
    Opensource
  • 3
    Very fast ingestion
  • 3
    Near real-time search
  • 2
    Low resource footprint
  • 2
    REST Api
  • 1
    Smart way of tagging
  • 1
    Perfect fit for k8s
  • 47
    Powerful easy to use monitoring
  • 38
    Flexible query language
  • 32
    Dimensional data model
  • 27
    Alerts
  • 23
    Active and responsive community
  • 22
    Extensive integrations
  • 19
    Easy to setup
  • 12
    Beautiful Model and Query language
  • 7
    Easy to extend
  • 6
    Nice
  • 3
    Written in Go
  • 2
    Good for experimentation
  • 1
    Easy for monitoring

Sign up to add or upvote prosMake informed product decisions

Cons of Loki
Cons of Prometheus
    Be the first to leave a con
    • 12
      Just for metrics
    • 6
      Bad UI
    • 6
      Needs monitoring to access metrics endpoints
    • 4
      Not easy to configure and use
    • 3
      Supports only active agents
    • 2
      Written in Go
    • 2
      TLS is quite difficult to understand
    • 2
      Requires multiple applications and tools
    • 1
      Single point of failure

    Sign up to add or upvote consMake informed product decisions

    What is Loki?

    Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate, as it does not index the contents of the logs, but rather a set of labels for each log stream.

    What is Prometheus?

    Prometheus is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true.

    Need advice about which tool to choose?Ask the StackShare community!

    What companies use Loki?
    What companies use Prometheus?
    See which teams inside your own company are using Loki or Prometheus.
    Sign up for StackShare EnterpriseLearn More

    Sign up to get full access to all the companiesMake informed product decisions

    What tools integrate with Loki?
    What tools integrate with Prometheus?

    Sign up to get full access to all the tool integrationsMake informed product decisions

    Blog Posts

    Dec 8 2020 at 5:50PM

    DigitalOcean

    GitHubMySQLPostgreSQL+11
    2
    2352
    May 21 2020 at 12:02AM

    Rancher Labs

    KubernetesAmazon EC2Grafana+12
    5
    1486
    PythonDockerKubernetes+14
    12
    2597
    Node.jsnpmKubernetes+6
    1
    1402
    What are some alternatives to Loki and Prometheus?
    Elasticsearch
    Elasticsearch is a distributed, RESTful search and analytics engine capable of storing data and searching it in near real time. Elasticsearch, Kibana, Beats and Logstash are the Elastic Stack (sometimes called the ELK Stack).
    ELK
    It is the acronym for three open source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine. Logstash is a server‑side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then sends it to a "stash" like Elasticsearch. Kibana lets users visualize data with charts and graphs in Elasticsearch.
    Log4j
    It is an open source logging framework. With this tool – logging behavior can be controlled by editing a configuration file only without touching the application binary and can be used to store the Selenium Automation flow logs.
    Castle Core
    It provides common Castle Project abstractions including logging services. It also features Castle DynamicProxy a lightweight runtime proxy generator, and Castle DictionaryAdapter.
    Bunyan
    It is a simple and fast JSON logging module for node.js services. It has extensible streams system for controlling where log records go (to a stream, to a file, log file rotation, etc.)
    See all alternatives