Kafka Gets KSQL JDBC Driver
Kafka Gets KSQL JDBC Driver
Written by Kay Ewbank   
Thursday, 16 November 2017

There's a new JDBC driver that provides standard access to Apache Kafka via JDBC API.  The current version connects to the KSQL engine to perform queries to Kafka and then, the engine translates those requests to Kafka requests. 

The sql-jdbc-driver is a Type 3 Java Database Connectivity (JDBC) driver. KSQL is an open source, Apache 2.0 licensed streaming SQL engine that enables stream processing against Apache Kafka.

KSQL is still a developer preview, and it has been designed to make it easier to read, write, and process streaming data in real-time, at scale, using SQL-like semantics. The developers say using SQL is an easy way to express stream processing transformations as an alternative to writing an application in a programming language such as Java or Python.

KSQL supports stream processing operations including aggregations, joins, windowing, and session management. KSQL doesn't let you do lookips on data; instead, it is used for continuous transformations. The example given by the developers is that of a stream of clicks from users along with a table of account information about those users being continuously updated. KSQL lets you model this stream of clicks, and table of users, and join the two together, even though the stream of clicks isn't a specific size.

The queries created using KSQL are described as continuous queries, transformations that run continuously as new data passes through them, on streams of data in Kafka topics.

The language does look SQL-like:

CREATE TABLE error_counts AS
SELECT error_code, count(*)FROM monitoring_stream
WINDOW TUMBLING (SIZE 1 MINUTE)
WHERE type = 'ERROR'

The developers suggest it could be used in applications where you need to transform an input stream into an output stream, but the application is simple enough to make the SQL interface enough without the need to use Kafka’s native streams API. KSQL uses Kafka’s Streams API internally and they share the same core abstractions for stream processing on Kafka.

Streams can be created from a Kafka topic or derived from existing streams and tables.

CREATE STREAM pageviews 
(viewtime BIGINT, userid VARCHAR, pageid VARCHAR) WITH (kafka_topic='pageviews', value_format=’JSON’);

While the current version of the JDBC driver just connects to the KSQL engine, and lets you run KSQL queries, the developers plan to add a standalone mode that will connect directly to Kafka brokers.


 ksql

 

 

More Information

KSQL JDBC Driver On GitHub

KSQL Website

Related Articles

Kafka 1 Becomes More Tolerant

Comparing Kafka To RabbitMQ

Apache Kafka Adds New Streams API

GoKa Stream Processing For Kafka

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, FacebookGoogle+ or Linkedin.

 

Banner


Atlas and the Backflip
18/11/2017

Atlas, Boston Dynamics' bipedal robot, has progressed from falling over to performing impressive gymnastics. There isn't much information on how it does it, but it is still worth watching.



Top 10 From Around the Web: More About Python
28/11/2017

This round up of Python-related items gathered from bog posts and external websites indicates the versatility of this popular language and has tips both for beginners and for more experienced Python p [ ... ]


More News

 

 
 

 

blog comments powered by Disqus

Last Updated ( Thursday, 16 November 2017 )
 
 

   
Banner
RSS feed of news items only
I Programmer News
Copyright © 2017 i-programmer.info. All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.