Apache Storm 2.0 has been released with better performance, new features, and integration with external systems. It has also had its core features rewritten in Java.

Apache Storm is a distributed real-time analytics system for Hadoop that can be used for processing large volumes of rapidly changing data. It can be used for realtime analytics, online machine learning, continuous computation, distributed RPC, and ETL. One advantage of Storm is its speed - a benchmark clocked it at over a million tuples processed per second per node.

One major change - though not a 'feature' - is that the core of Storm has been re-implemented in Java. Until now it was written in Clojure. Rewriting in Java has improved performance significantly, and the developers say it has made Storm's internal APIs more maintainable and extensible. They also hope it will make potential contributors more willing to work on it.

The performance improvements also come from a leaner threading model, a new fast messaging subsystem and a lightweight back pressure model. The developers say Storm 2.0 is the first streaming engine to break the 1 microsecond latency barrier.

There's a new streams API that is typed and can be used to express streaming computations more easily using functional style operations. It builds on top of the Storm's core spouts and bolt APIs and automatically fuses multiple operations together to optimize the pipeline.

Support for data windowing has also been improved. The Windowing API can now save and restore the window state to the configured state backend so that larger continuous windows can be supported. The window boundaries can now be accessed via the APIs.

Another major change in the way Storm is implemented comes with the removal of Storm-Kafka. This was used for Kafka integration, but was deprecated due to Kafka's deprecation of the underlying client library. Users will have to move to the storm-kafka-client module, which uses Kafka's ´kafka-clients´ library for integration.

The new release of Storm is available on the Storm project website. 

More Information

Storm Website

Related Articles

Apache Storm On Azure

Apache Storm Added To Hadoop On Azure  

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info