Apache Ignite Adds Change Data Capture
Written by Kay Ewbank   
Thursday, 20 January 2022

Apache Ignite has been updated with improvements including Change Data Capture (CDC), an Index Query API, and several vulnerability fixes. Ignite is a distributed database for high-performance computing with in-memory speed.

Ignite can be used as a traditional SQL database via JDBC drivers, ODBC drivers, or its own native SQL APIs. By default, it runs purely in-memory, but clusters can be configured to run on a mix of disk and memory.

ignite

It supports co--located compute in Java, Scala, Kotlin, C#, and C++.

Change Data Capture (CDC) is a data processing pattern used to asynchronously receive entries that have been changed on the local node so that action can be taken using the changed entry.

The Ignite team says CDC is useful to handle streaming changes in a data warehouse, updating search indexes, and async interactions with external systems. It can also be used to calculate statistics for streaming queries. Ignite implements CDC with the ignite-cdc.sh application and Java API.

When CDC is enabled, the Ignite server node creates a hard link to each WAL archive segment in a special directory. The ignite-cdc.sh application can be runruns on a different JVM and processes newly archived WAL segments. When the segment is fully processed by ignite-cdc.sh, it is removed. WAL (write ahead log) archive segments hold the details of all data modifying operations (including deletes) that happen on a node.

Another improvement in this release is the provision of an Index Query API for existing indexes. Index queries work over distributed indexes and retrieve cache entries that match the specified query. The new API should help when SQL is not applicable because of the design of an application, or where IndexScan is preferable to ScanQuery for performance reasons.

Apache Ignite 2.12 is available now. ignite

More Information

Apache Ignite Website

Related Articles

Apache Ignite Adds Spark DataFrames Support

Spark 3 Improves Python and SQL Support

Apache Superset Reaches Top Level Project Status

Apache Daffodil Now Top Level Project

Facebook Apollo NoSQL Database 

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Android 15 Developer Preview Released
19/02/2024

Android 15 Developer Preview has just been released by the Android team with features including partial screen sharing and the latest version of the Privacy Sandbox.



SnapCode: A Java IDE for the Web
27/02/2024

Thanks to CheerpJ and WebAssembly you can now run a Java IDE inside your browser and local first.This is SnapCode, and while lightweight and in-browser, is to be not underestimated.


More News

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info