Apache Arrow 16 Adds Azure Blob Support
Written by Kay Ewbank   
Tuesday, 21 May 2024

Apache Arrow 16 has been released with improvements to the C data interface and to Arrow Flight RPC, and the addition of support for Azure Blob data format.

Apache Arrow is a development platform for in-memory analytics. It has technologies that enable big data systems to process and move data quickly. Arrow is language independent, can be used for flat and hierarchical data, and the data store is organized for efficient analytic operations. It also provides computational libraries. Languages currently supported are C, C++, C#, Go, Java, JavaScript, Julia, MATLAB, Python, R, Ruby, and Rust.

arrow

The improvements to the new release start with the addition of initial support for the Azure Blob Storage to the C++ library in Arrow. Azure Blob storage provides scalable, cost-efficient object storage in the cloud and can be used to store and access unstructured data. Other improvements to the C++ support in Arrow include the ability to build Arrow C++ with Emscripten, which lays the foundation for running Arrow C++ under WASM runtimes and eventually PyArrow as well. Support has also been added to convert from table and RecordBatch to a tensor.

Arrow Flight RPC now has support for session variable RPCs, and the Go support in Flight handles existing credentials correctly across connections.

Improvements to the Python support in Arrow include the addition of formerly missing methods to RecordBatch, and added support for Binary and StringView in PyArrow. Support has also been added for Run-End Encoded arrays in PyArrow.

The C Data Interface now manages mappings between a device type and id to a memory manager; and there's support for handling Chunked Arrays in the C Stream Interface.

More generally in Arrow Flight RPC,session variable RPCs have been added and the JDBC driver and Java features have been improved.

Apache Arrow 16 is available now.  

arrow

More Information

Apache Arrow Website

Related Articles

Apache Arrow 15 Stabilizes Flight SQL

Apache Arrow Improves C++ Support

Apache Arrow 5 Improves Asynchronous Scanner

Apache Arrow 4 Adds New C++ Compute Functions

Apache Arrow Improves C++ Support

Apache Arrow 2 Improves C++ and Rust Support

Apache Arrow Reaches 1.0

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Second Gen Robot Dog On Kickstarter
15/09/2024

We covered Mini-Pupper's launch back in 2021 and now there is an even better, second generation, of this low-cost and capable quadruped robot. If you want to see what AI can bring to a small robot, th [ ... ]



Pulumi Envisions The Intelligent Cloud
18/09/2024

Today at PulumiUp, its 4th annual conference, Pulumi is announcing "Intelligent Cloud", its vision for AI-powered cloud infrastructure, and two new products, ESC GA and Insights 2.0.  


More News

kotlin book

 

Comments




or email your comment to: comments@i-programmer.info