Acryl Adds Advanced Data Observability
Written by Kay Ewbank   
Monday, 26 June 2023

Acryl Data has announced a new module for its data catalog platform. Acryl Observe will give users a single platform for data observability, governance, and discovery. Acryl Data is the main company behind the open source DataHub Project, the most widely-used open source metadata platform and data catalog and largest metadata community in the world.

The DataHub project is based on the DataHub metadata platform which was originally built at LinkedIn and subsequently open-sourced under the Apache 2.0 License. The project is described as a third generation metadata platform that enables data discovery, collaboration, governance, and end-to-end observability. Acryl Data is a major supporter of the project.

acrylsq

Acryl Observe is a new module for Acryl's platform. The developers say it is driven by a central metadata control plane, and continuously detects data quality incidents in real-time, automates anomaly detection to prevent breakages and drives faster resolution when they do occur.

acryl observe

The automatic detection of anomalies aims to prevent problems with data quality breakages, and to sort out problems more rapidly when something does go wrong. The developers say they propose data contracts for producers to review and accept, so that accountability is clearer.

The module offers a high-level view of the health of an organization’s data, so data managers can monitor the health of all datasets in real-time, with automated and user-defined assertions for data quality.

Pipelines and tasks can also be monitored in real-time, with automated checks on the health and analysis of downtime. The module can also be used to track what has been affected by incidents, so reducing time-to-restart on failed jobs. The dashboard can also be used to explore data tables to find active incidents, failing assertions, and "pending anomalies".

acryl observe

Acryl says the new module can also be used to build continuously-monitored SLAs for tasks, datasets, tables and columns, and enforce automated workflows that trigger when assertions fail.

Acryl Observe is available now on the Acryl Data platform.

acrylsq

 

More Information

Acryl Data Website

DataHub Project

Related Articles

Fiberplane Adds Metrics Tool

Google Builds Data Lake On BigQuery

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Azure AI And Pgvector Run Generative AI Directly On Postgres
26/03/2024

It's a match made in heaven. The Azure AI extension enables the database to call into various Azure AI services like Azure OpenAI. Combined with pgvector you can go far beyond full text search. Let's  [ ... ]



Can C++ Be As Safe As Rust?
10/04/2024

Herb Sutter is a well known and respected C++ champion and he thinks that the language only needs a few tweaks to make it as safe as Rust. Can this be true?


More News

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info