Microsoft Showcases Key Value Store
Written by Kay Ewbank   
Tuesday, 12 June 2018

Researchers from Microsoft have presented a new embedded key-value store called FASTER at SIGMOD 2018.


faster

 

The key value store is innovative in that it supports the use of in-place updates, meaning it can support fast and frequent lookups and updates of large amounts of state information. The developers say that they see this as being particularly useful in scenarios such as Internet-of-Things where billions of devices report and update state such as per-device performance counters.

At the moment, if an application needs to maintain such state, it will typically scale out on multiple machines for memory, under utilizing other resources such as storage and networking on the machine. Such applications tend to have temporal locality - data or resources are reused within a small time frame. Faster makes use of this to control the in-memory footprint of the system and cache the frequently accessed values without maintaining any fine-grained statistics per record.

It does this with a combination of an in-memory allocator that handles data access and in-place updates to records, alongside an append-only log-structured allocator that can handle data larger than main-memory, but without in-place updates.

Faster has two technical innovations. Firstly, its hash index is cache-friendly and concurrent latch-free, and is designed to grow and shrink dynamically while maintaining logical pointers to records in a log. The second change is a concurrent hybrid log record allocator that returns logical or physical memory pointers, and works alongside the index so spans fast storage (such as cloud storage and SSD) and main memory.

The head of the hybrid log is located in storage, and uses a read-copy-update strategy for updating records. The tail of the hybrid log is located in main memory and uses in-place updates. In between these two regions lies a read-only region in memory that provides hot records a “second chance” to be quickly copied back to the tail.

This mix of locations and techniques utilizes the temporal locality of updates, allows records to spill to sequential storage efficiently and enables a natural clustering of hot records in memory for fast in-place updates.

The researchers say that Faster can outperform even pure in-memory data structures such as the Intel TBB hash map when the working set fits in memory. They say it also outperforms today’s key-value stores and caching systems such as RocksDB and Redis by several orders-of-magnitude.

faster

 

More Information

PDF Of Conference Paper

Related Articles

RocksDB on Steroids

RocksDB - Facebook's Database Now Open Source

Microsoft Expands Redis Support

Apple Open Sources FoundationDB

 

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, Facebook or Linkedin.

Banner


Tesla Model 3 Pwn2Own'd
29/03/2019

Team Fluoroacetate was the only one to attempt to hack the Tesla Model 3 at the recent Pwn2Own contest held in conjunction with the annual CanSecWest security contest earlier this month. Their hack wa [ ... ]



Wing Python Improves Code Warnings
12/04/2019

There's a new version of Wing Python. Version 7 has an improved code warnings and code quality inspection system that includes built-in error detection and tight integration with pylint, pep8, and myp [ ... ]


More News

Python

 



 

Comments




or email your comment to: comments@i-programmer.info