Apache Avro Adds ZStandard Support
Written by Kay Ewbank   
Friday, 31 May 2019

Apache Avro 1.9 is available, with an updated JSON reader, smaller size, and support for ZStandard compression.

avro

Avro is a data serialization framework for high volume, high performance, high throughput, data processing systems. It uses JSON for defining data types and protocols, and serializes data in a compact binary format. Avro was developed by the same team that developed Hadoop, and it is used by data processing systems including Hadoop, Spark and Kafka.

Avro can be used as a toolkit via remote procedure calls (RPCs). The reference implementation of Avro is developed and released as a Java library. Avro has language binding support for C, C++, C#, Go, Haskell, Java, Perl, PHP, Python, Ruby, Scala, and TypeScript. It uses a fast and compact binary data format, has a language independent schema definition format. and uses a container file to store and persist data.

The improvements to the new release are designed to make it leaner and faster. The changes begin with the fact that the new version is built with Java 8 by default. This means the Joda time library is no longer needed for handling logical date and time values, as Java 8 handles dates and times natively.

The next improvement changes the way JSON reading is done. In previous versions this was carried out using Java's Jackson library. In this release the old Codehaus Jackson has been replaced by FasterXML's Jackson 2.9. In addition to being faster, this has security improvements. The developers have also removed Jackson classes from public API. 

Support for Facebook's ZStandard compression has also been added. ZStandard is a real-time compression algorithm, providing high compression ratios.

Elsewhere, in the drive to make Acro leaner, multiple dependencies were removed, including guava, paranamer, commons-codec, and commons-logging. Apache Avro is compiled and tested with Java 11.

avro

More Information

Avro Website

Related Articles

KSQL Adds Avro Support

Hadoop 3 Adds HDFS Erasure Coding

Serialization Will Go From Java - Sometime  

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, Facebook or Linkedin.

Banner


NOOA Datasets Now On Google Cloud
14/01/2020

A collaboration between Google and The U.S. National Oceanic and Atmospheric Administration (NOAA) has added many new datasets to those available on Google Cloud. The datasets have been added as part  [ ... ]



IntelliJ IDEA Updated To Be Faster And Smaller
31/12/2019

There's a new release of JetBrains IntelliJ IDEA, that the developers claim is faster to start up, uses less memory, and has a more responsive UI. IntelliJ IDEA is a well-known Java IDE for web, deskt [ ... ]


More News

graphics

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Friday, 31 May 2019 )