Google has released a new Python client library for Data Commons based on the V2 REST API. They say the new library enhances how data developers can make use of Data Commons.

Data Commons is a Google initiative that brings together the world's publicly available statistical data with the aim of making it more accessible and useful for everyone. It's open-source and contains public data from a wide range of sources. The Data Commons data can be used by developers, researchers, and data analysts, and Google Search uses Data Commons to answer queries and generate the "top" graphs. 

The Python client library is version two of an earlier release, and Google says one addition is robust support for custom instances. This means you can now use the Python library to programmatically query any public or private instance, whether hosted locally, within your organization or on the Google Cloud Platform.

The Data Commons APIs allow developers to programmatically access the data in Data Commons, using either a REST API that can be used from the command line or from languages with an HTTP library; or via a Python client library that wraps the REST APIs and includes support for Pandas (as in this library).

There are four endpoints: statistical data, graph exploration, data within the Data Commons data sets, and graph query/SPARQL.

The statistical data will return observations when given a set of statistical variables, dates and entities. Graph exploration accepts inputs of a set of nodes, and can be used to explore the graph around those nodes. 

The way the graph query/SPARQL option works is that given a subgraph where some of the nodes are variables, this options retrieves possible matches. This corresponds to a subset of the graph query language SPARQL, and is aimed at use for complex node connections which would require multiple API calls; the developers give the example of "hate crimes motivated by disability status in Californian cities".

The Python library makes it easy to perform common queries against Data Commons data, such as exploring the structure of the knowledge graph; retrieving data for any of the 200,000+ statistical variables from over 200 datasets; and mapping entities from other datasets to entities in Data Commons.

V2 of the client library offers many technical improvements over the earlier version. It now supports Pandas dataframe APIs as an integral module, with a single installation package. This allows the APIs to be used with other API endpoints in the same client. 

There are also several new convenience methods for common data queries; and API key management and other stateful operations are built in to the client class. 

The library is integrated with the Pydantic libraries for improved type safety, validation and serialization; and it now has support for multiple response formats, including JSON and Python dictionaries and lists. 

To get started with the Data Commons Python library, you can install the package directly from PyPI. The developers have also provided resources to help get started, including reference documentation and online tutorials available as Google Colab notebooks. The open-source code is also available on GitHub.

More Information

Data Commons Website

Data Commons Python Client On GitHub

Related Articles

Google Announces BigQuery Metastore

Google Adds Ability To See Datasets

Google Announces Framework For Data Science Predictions

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info