Apache Arrow Improves C++ Support
Written by Kay Ewbank   
Thursday, 11 August 2022

Apache Arrow has been updated with better support for C++, extensions to Flight SQL, and substantial improvements to the Parquet crate.

Apache Arrow is a development platform for in-memory analytics. It has technologies that enable big data systems to process and move data fast..It is language independent, can be used for flat and hierarchical data, and the data store is organized for efficient analytic operations. It also provides computational libraries. Languages currently supported are C, C++, C#, Go, Java, JavaScript, Julia, MATLAB, Python, R, Ruby, and Rust.

arrow

The improvements to the new release start with Arrow Flight. Flight SQL has been extended with a method to get type metadata and column metadata in returned schemas.

The C++ libraries now support UCX as a network transport, and the APIs have been reworked so other transports can be implemented, and to use the arrow::Result type. You also now have the option of building Arrow C++ with support for the experimental Substrait query representation format. Substrait is a project that aims to offer cross-language serialization for relational algebra.

Support for temporal data is another area to have improvements, with a number of new compute kernels for operating on temporal data, so you can add, subtract and multiply between various temporal types, and new functions to check for daylight saving time and leap years.

The Arrow parquet crate has also received substantial improvements in this release to improve the raw performance for reading and writing mirroring the efforts that went into the C++ version a few years ago. The API has also been improved so it is easy to use efficiently with modern Rust for two reading from local disks and reading asynchronously from remote object stores.

Apache Arrow is available for download.

arrow 

More Information

Apache Arrow Website

Arrow On GitHub

Substrait Website

Related Articles

Apache Arrow 5 Improves Asynchronous Scanner

Apache Arrow 4 Adds New C++ Compute Functions

Apache Arrow Improves C++ Support

Apache Arrow 2 Improves C++ and Rust Support

Apache Arrow Reaches 1.0

 

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


GitHub Completes Arctic Vault Drop
20/09/2022

GitHub has added yet more material to the Arctic Vault in Svalbard, and says this completes its part of the task. The vault now contains the 02/02/2020 snapshot of every active public GitHub repositor [ ... ]



Linux And Android Waste Coding Effort
14/09/2022

For many years it has been standard practice to test that you get the memory you ask for, but it has all be a huge waste of time. Operating systems get in on the act before you have a chance to do any [ ... ]


More News

pythondata

 



 

Comments




or email your comment to: comments@i-programmer.info