Google Announces BigQuery Metastore
Thursday, 30 January 2025

Google has announced the public preview of BigQuery metastore, a fully managed, unified metadata service that works with multiple engines and supports consistent data governance.

Google describes BigQuery metastore as a highly scalable runtime metadata service that works with multiple engines including BigQuery, Apache Spark, Apache Hive and Apache Flink, and supports the open Apache Iceberg table format.

bigquery

Google BigQuery is a distributed, serverless SQL engine that provides a way to query petabytes of data. It has built in machine learning, is serverless, and supported by Google Cloud.

The new service is designed so analytics engines can work with and query one copy of the data with a single schema, whether the data is stored in BigQuery storage tables, BigQuery tables for Apache Iceberg, or BigLake external tables.

Google says the BigQuery metastore will serve as a critical component for customers looking to migrate and modernize from legacy data lakes to a modern lakehouse architecture. It is closely integrated with BigQuery, and provides built-in security and governance for user interactions with data.

Google says the key benefits of BigQuery metastore are its cross-engine interoperability, support for open formats and catalogs, built-in governance, and the fact it is fully managed at BigQuery scale.

The cross-engine interoperability means it provides a unified view of all metadata for all data sources in the lakehouse, which enables query processing and DML for data stored in open and proprietary formats across object stores, BigQuery storage, and across analytics runtimes.

The built-in governance comes from the fact it is with the governance capabilities provided in BigQuery, such as automated cataloging and universal search, business metadata, data profiling, data quality, fine-grained access controls, data masking, sharing, data lineage and audit logging.

Being a serverless, fully managed service, BigQuery metastore has integrations with engines including BigQuery, Spark, Hive and Flink.

Google BigQuery Metastore is available in preview now.

 bigquery

More Information

Google BigQuery

Related Articles

Google BigQuery Updated

BigQuery Updated and Repriced

Google BigQuery Updated

BigQuery Now Open to All

Google BigQuery Service

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Kotlin Multiplatform Drops Support For Fleet
13/02/2025

JetBrains has announced that it is abandoning its plan to create a standalone Kotlin Multiplatform IDE and is planning to deprecate support for KMP on its lightweight Fleet IDE in order to focus its e [ ... ]



Apache Hudi 1.0 Released
21/01/2025

Apache has released Hudi 1.0, described as a landmark achievement that defines what the next generation of data lakehouses should achieve. Hudi pioneered transactional data lakes in 2017. 


More News

espbook

 

Comments




or email your comment to: comments@i-programmer.info