Grafana Releases Loki 3
Written by Alex Denham   
Tuesday, 07 May 2024

Grafana has announced the release of Loki 3, with improvements including query acceleration with Bloom filters and native OpenTelemetry support.

Loki is a horizontally scalable, highly available, multi-tenant log aggregation system inspired by Prometheus. It consists of a set of components that can be composed into a fully featured logging stack.

grafana

Compared to other log aggregation systems, Loki does not do full text indexing on logs. Instead, it stores compressed, unstructured logs and only indexes metadata, making it simpler to operate and cheaper to run. It also indexes and groups log streams using the same labels used by Prometheus, so users can switch between metrics and logs using the same labels.

The main headline improvement to this version of Loki is query acceleration with Bloom filters, an experimental feature designed to help users find specific log data faster.

The developers say that previous releases of Loki were good at finding log data very quickly given information about roughly where to look such as label selectors to filter on. The addition of Bloom filters to this release are designed for searches where it isn't possible to use filters. They say that until now, if a user wanted to find one particular message in a log, Loki had to look every log line within all matching streams. Bloom filters add the ability to search for strings, such as order ID or user ID, skipping large chunks of data that don't contain the information you're seeking. They say that early internal tests suggest that with Bloom filters, Loki can skip a significant percentage of log data while running queries:

"Our dev environment tests show that we can now filter out between 70% and 90% of the chunks we previously needed to process a query"

A bloom filter is a probabilistic data structure that is based on hashing, named after the inventor, Burton Howard Bloom. 

The second main change is support for OpenTelemetry, which the Loki team say 54% of the respondents in Grafana's 2024 Observability Survey said they were using more than they did last year. OpenTelemetry aims to provide a vendor-agnostic way to easily instrument applications or systems, regardless of their language, infrastructure, or runtime environment.

Until now, if you wanted to view OpenTelemetry format logs in Loki, they had to be ingested using Loki Exporter which serialized the data in JSON, meaning it then had to be deserialized at query time to interact with OTel attributes and metadata in log records.

The new version has native OpenTelemetry ingestion support, avoiding the need for these stages.

Loki 3 is available now.

grafana

More Information

Grafana Website

Loki On GitHub

Related Articles

Grafana 11 Improves Metrics

What's the Best Way to Effectively Monitor a Kubernetes Cluster?

Grafana Adds New Tools

Grafana Ships Grafana 10

Grafana Updates Cloud Platform

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Microsoft's Cybersecurity For Beginners
30/04/2024

A free, self-paced course about Cybersecurity 101 is on offer by Microsoft's Cloud Security Advocates. It's a 30+ lesson curriculum targeted at complete novices.



GitLab Adds Google Cloud Integration
14/05/2024

GitLab has released public betas of the integration features with Google Cloud that the company announced in 2023. The integration means GitLab’s DevSecOps workflow integrates with Google Cloud secu [ ... ]


More News

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Tuesday, 07 May 2024 )