Fast Data Requires New Frameworks
Written by Janet Swift   
Monday, 25 September 2017

Developers are adopting new streaming data frameworks and turning to microservices to meet the need to use data faster according to a report from Lightbend presented at today's Strata Data Conference being held in New York.

fastdatabanner 

The premise which underlies the report starts is that the explosive growth in data, made possible by emerging technologies, has:

huge implications for how enterprises interact with data to create future business opportunities.

To explore the impact of so-called "Fast Data", Lightbend, creator of the Scala language that Apache Spark and many of the other most-used frameworks in big data are written in, surveyed 2,457 developers to examine adoption trends.

Commenting on the adoption of fast data, Mark Brewer, CEO of Lightbend told IProgrammer:

To compete in the digital era, the necessity is rising for enterprises to use data faster. This need for speed is expanding beyond analytics to applications that adapt to changing conditions in real-time, personalize customer engagement, and power the internet of everything.” 
 

The three key findings presented in the report are:

  • Fast data has clear business value - overall 60% of developers say their senior management understands the value of fast data although the pattern across indutries varies - insurance, construction and government being less likely to, which biotechnology and agriculture leading the field.

fd1

  • Enterprises will need batch and streaming for fast data -developers say 90% of their data processing workloads include a real-time component.

  • Developers are choosing new frameworks and languages based on fast data requirements.

Adoption of fast data technology appears to be happening three times faster than that of Hadoop and its biggest hurdle is in choosing the right tools and techniques, with the design phase of the software development lifecycle being the most problematic

 

fd2

 

Lightbend reports that while fast data tools are emerging at a rapid rate some are more on developers' rader than others, with Kafka leading in terms of current use:

 

fd3

It also argues that a key characteristic of fast data architectures is the use of microservices for streaming applications and reveals that 75% of developers with advanced fast data use cases in production rely on microservices:

fd4

The survey also found that as fast data use cases move into production microservices were increasingly valued for DevOps agility and predicatability.

In sum, data requirement are influencing the selection of techology with 37% of developers answering:

We’re choosing new frameworks based on their ability to handle data more effectively

and 18% answering:

We’re choosing new languages based on their ability to handle data more effectively 

 fdsq

 

 

More Information

Fast Data Development Trends 2017 

Lightbend

Strata Data Conference

Related Articles

Apache Kafka Adds New Streams API 

Apache Spark 2.0 Released

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

 

Banner


PlanetScale Gets Into Vector Search
02/12/2024

PlanetScale, the cloud MySQL-compatible database with advanced scaling capabilities, is now upgraded with vector storage and search.



Go At Highest Rank Ever in TIOBE Index
20/11/2024

Go is currently in 7th place in the TIOBE Index for November 2024. Not only is this is the highest position it has ever had, it's percentage rating is almost equal to its all-time-high. Will Go contin [ ... ]


More News

espbook

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Tuesday, 26 September 2017 )