Major Update to Google BigQuery
Written by Kay Ewbank   
Monday, 23 September 2013

Google has announced a major update to BigQuery, adding real-time support to its service for analyzing large amounts of data.

BigQuery has already been updated this year to provide support for big scale JOINs and GROUP BYs and unlimited result sizes. The new improvements add the option of querying subsets of the latest data, more functions and browser tool improvements alongside the ability to run queries in real time. The browser has been improved with a buttons for common tasks in the query history panel, and more information made visible about the queries.

 

(click in screenshot for larger version)

 

The real-time support has been added through a simple API call, tabledata().insertAll(), that lets you store data as it comes in and query it instantly. Talking about the improvements on the Google Developer’s Blog, Felipe Hoffa, Developer Programs Engineer at Google said this feature is ideal for time sensitive use cases like log analysis and alerts generation, and that you can use it simply by calling the new endpoint with your data in a JSON object (with a single row or multiple rows of data).

Streaming data into BigQuery is free until January 1st, 2014, after when it will be billed at a flat rate of 1 cent per 10,000 rows inserted. The traditional jobs().insert() method will continue to be free.

The second improvement is the ability to define queries that only scan a range or spot in the previous 24 hours. Traditionally BigQuery has always done a "full column scan" when querying data, while the new syntax will allow you to focus only on a specific subset of the latest data, so lowering the costs of queries. You can query only the last hour of inserted data, or what was inserted before that hour, or get a snapshot of the table at a specific time.

Google has also added new window functions, namely SUM(), COUNT(), AVG(), MIN(), MAX(), FIRST_VALUE, and LAST_VALUE(), along with the statistical functions COVAR_POP(), COVAR_SAMP(), STDDEV_POP(), STDDEV_SAMP(), VAR_POP() and VAR_SAMP(). Window functions let you carry out calculations on a specific partition, or "window", of a result set.

 

 

More Information

Google BigQuery goes real-time

Google BigQuery

 

Related Articles

BigQuery Updated and Repriced

Google BigQuery Updated

BigQuery Now Open to All

Google BigQuery Service

 

To be informed about new articles on I Programmer, install the I Programmer Toolbar, subscribe to the RSS feed, follow us on, Twitter, Facebook, Google+ or Linkedin,  or sign up for our weekly newsletter.

 

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

 

Banner


Google Adds Multiple Database Support To Firestore
04/03/2024

Google has announced the general availability of Firestore Multiple Databases, which can be used to manage multiple Firestore databases within a single Google Cloud project.



Bun Shell Released
29/02/2024

The developers of the Bun JavaScript runtime have released Bun Shell, a new experimental embedded language and interpreter in Bun that lets you run cross-platform shell scripts in JavaScript and TypeS [ ... ]


More News

 

Last Updated ( Monday, 23 September 2013 )