Apache Ignite Adds Change Data Capture
Written by Kay Ewbank   
Thursday, 20 January 2022

Apache Ignite has been updated with improvements including Change Data Capture (CDC), an Index Query API, and several vulnerability fixes. Ignite is a distributed database for high-performance computing with in-memory speed.

Ignite can be used as a traditional SQL database via JDBC drivers, ODBC drivers, or its own native SQL APIs. By default, it runs purely in-memory, but clusters can be configured to run on a mix of disk and memory.

ignite

It supports co--located compute in Java, Scala, Kotlin, C#, and C++.

Change Data Capture (CDC) is a data processing pattern used to asynchronously receive entries that have been changed on the local node so that action can be taken using the changed entry.

The Ignite team says CDC is useful to handle streaming changes in a data warehouse, updating search indexes, and async interactions with external systems. It can also be used to calculate statistics for streaming queries. Ignite implements CDC with the ignite-cdc.sh application and Java API.

When CDC is enabled, the Ignite server node creates a hard link to each WAL archive segment in a special directory. The ignite-cdc.sh application can be runruns on a different JVM and processes newly archived WAL segments. When the segment is fully processed by ignite-cdc.sh, it is removed. WAL (write ahead log) archive segments hold the details of all data modifying operations (including deletes) that happen on a node.

Another improvement in this release is the provision of an Index Query API for existing indexes. Index queries work over distributed indexes and retrieve cache entries that match the specified query. The new API should help when SQL is not applicable because of the design of an application, or where IndexScan is preferable to ScanQuery for performance reasons.

Apache Ignite 2.12 is available now. ignite

More Information

Apache Ignite Website

Related Articles

Apache Ignite Adds Spark DataFrames Support

Spark 3 Improves Python and SQL Support

Apache Superset Reaches Top Level Project Status

Apache Daffodil Now Top Level Project

Facebook Apollo NoSQL Database 

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Android Studio Jellyfish Ready To Use
08/05/2024

Well, as ready as any of the recent Android Studio's have been. This one boasts an AI assistant called Gemini - shame Android Studio isn't as fast to implement as Gemini is to suggest.



OpenSilver 2.2 Adds LightSwitch Compatibility Pack
30/04/2024

OpenSilver 2.2 has been released with the addition of a LightSwitch Compatibility Pack designed to provide a way to run legacy Visual Studio LightSwitch applications on modern browsers. The open-sourc [ ... ]


More News

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info