Fast Data Requires New Frameworks
Fast Data Requires New Frameworks
Written by Janet Swift   
Monday, 25 September 2017

Developers are adopting new streaming data frameworks and turning to microservices to meet the need to use data faster according to a report from Lightbend presented at today's Strata Data Conference being held in New York.


The premise which underlies the report starts is that the explosive growth in data, made possible by emerging technologies, has:

huge implications for how enterprises interact with data to create future business opportunities.

To explore the impact of so-called "Fast Data", Lightbend, creator of the Scala language that Apache Spark and many of the other most-used frameworks in big data are written in, surveyed 2,457 developers to examine adoption trends.

Commenting on the adoption of fast data, Mark Brewer, CEO of Lightbend told IProgrammer:

To compete in the digital era, the necessity is rising for enterprises to use data faster. This need for speed is expanding beyond analytics to applications that adapt to changing conditions in real-time, personalize customer engagement, and power the internet of everything.” 

The three key findings presented in the report are:

  • Fast data has clear business value - overall 60% of developers say their senior management understands the value of fast data although the pattern across indutries varies - insurance, construction and government being less likely to, which biotechnology and agriculture leading the field.


  • Enterprises will need batch and streaming for fast data -developers say 90% of their data processing workloads include a real-time component.

  • Developers are choosing new frameworks and languages based on fast data requirements.

Adoption of fast data technology appears to be happening three times faster than that of Hadoop and its biggest hurdle is in choosing the right tools and techniques, with the design phase of the software development lifecycle being the most problematic




Lightbend reports that while fast data tools are emerging at a rapid rate some are more on developers' rader than others, with Kafka leading in terms of current use:



It also argues that a key characteristic of fast data architectures is the use of microservices for streaming applications and reveals that 75% of developers with advanced fast data use cases in production rely on microservices:


The survey also found that as fast data use cases move into production microservices were increasingly valued for DevOps agility and predicatability.

In sum, data requirement are influencing the selection of techology with 37% of developers answering:

We’re choosing new frameworks based on their ability to handle data more effectively

and 18% answering:

We’re choosing new languages based on their ability to handle data more effectively 




More Information

Fast Data Development Trends 2017 


Strata Data Conference

Related Articles

Apache Kafka Adds New Streams API 

Apache Spark 2.0 Released

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, FacebookGoogle+ or Linkedin.



Explore Computer History With BBC Archive

Providing a remarkable trip down memory lane for anyone who learned to program in 1980s Britain, the BBC has released an archive of its Computer Literacy Project.

All A Flutter - Google's Cross Platform App Framework Ready To Go

Google's Flutter is about to be released as a Version 1 product, ready for you to use. The question is should you be interested in a way to create almost native apps that run on iOS or Android? I supp [ ... ]

More News





or email your comment to:

Last Updated ( Tuesday, 26 September 2017 )

RSS feed of news items only
I Programmer News
Copyright © 2018 All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.