| Apache Ignite Adds Change Data Capture |
| Written by Kay Ewbank |
| Thursday, 20 January 2022 |
|
Apache Ignite has been updated with improvements including Change Data Capture (CDC), an Index Query API, and several vulnerability fixes. Ignite is a distributed database for high-performance computing with in-memory speed. Ignite can be used as a traditional SQL database via JDBC drivers, ODBC drivers, or its own native SQL APIs. By default, it runs purely in-memory, but clusters can be configured to run on a mix of disk and memory.
It supports co--located compute in Java, Scala, Kotlin, C#, and C++. Change Data Capture (CDC) is a data processing pattern used to asynchronously receive entries that have been changed on the local node so that action can be taken using the changed entry. The Ignite team says CDC is useful to handle streaming changes in a data warehouse, updating search indexes, and async interactions with external systems. It can also be used to calculate statistics for streaming queries. Ignite implements CDC with the ignite-cdc.sh application and Java API. When CDC is enabled, the Ignite server node creates a hard link to each WAL archive segment in a special directory. The ignite-cdc.sh application can be runruns on a different JVM and processes newly archived WAL segments. When the segment is fully processed by ignite-cdc.sh, it is removed. WAL (write ahead log) archive segments hold the details of all data modifying operations (including deletes) that happen on a node. Another improvement in this release is the provision of an Index Query API for existing indexes. Index queries work over distributed indexes and retrieve cache entries that match the specified query. The new API should help when SQL is not applicable because of the design of an application, or where IndexScan is preferable to ScanQuery for performance reasons. Apache Ignite 2.12 is available now. More InformationRelated ArticlesApache Ignite Adds Spark DataFrames Support Spark 3 Improves Python and SQL Support Apache Superset Reaches Top Level Project Status Apache Daffodil Now Top Level Project Facebook Apollo NoSQL Database
To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin. 
Comments
or email your comment to: comments@i-programmer.info |
