RocksDB - Facebook's Database Now Open Source
RocksDB - Facebook's Database Now Open Source
Written by Kay Ewbank   
Thursday, 21 November 2013

The Facebook Database Engineering Team has open-sourced RocksDB, an embeddable persistent key-value store for fast storage that’s based on Google’s LevelDB.

RocksDB is aimed at embedded workloads, though the team says it can also be the foundation for a client-server database. RocksDB builds on LevelDB to be scalable to run on servers with many CPU cores. It is designed to use fast storage efficiently, and to support IO-bound, in-memory and write-once workloads.




The Rocksdb project started at Facebook as an experiment to develop an efficient database to use flash storage for server workloads. It is a C++ library and can be used to store keys-and-values where keys and values are arbitrary size byte streams, with support for atomic reads and atomic writes. RocksDB can be tuned to run on data on pure memory, flash, hard disks or on HDFS. It has support for various compression algorithms and good tools for production support and debugging.

The Facebook team began by benchmarking LevelDB, but it was unsuitable for the Facebook server workloads.

According to the RocksDB team: 

“The benchmark results look awesome at first sight, but we quickly realized that those results were for a database whose size was smaller than the size of RAM on the test machine - where the entire database could fit in the OS page cache. When we performed the same benchmarks on a database that was at least 5 times larger than main memory, the performance results were dismal.“

Having developed RocksDB and devised benchmarks to test its for server-workload on Flash, the Facebook team were able to show that RocksDB "solidly outperforms" LevelDB. The team comments:

“We found that LevelDB's single-threaded compaction process was insufficient to drive server workloads. We saw frequent write-stalls with LevelDB that caused 99-percentile latency to be tremendously large. We found that mmap-ing a file into the OS cache introduced performance bottlenecks for reads. We could not make LevelDB consume all the IOs offered by the underlying Flash storage.”

The benchmarks and their results are reported on GitHub. RockDB's code is also available on Github. Facebook Engineer Dhruba Borthakur gave introductory talk on RocksDB at the Data @ Scale 2013 conference, and his slides are available to view.



Suggested uses of RocksDB are applications that need low latency database accesses, such as a user-facing application that stores the viewing history and state of users of a website, or a spam detection application that needs fast access to big data sets. RocksDB can also be used to cache data from Hadoop, thereby allowing applications to query Hadoop data in real time.

More Information

RocksDB on Github

Related Articles

MariaDB 10.0 Beta Introduces Replication Enhancements

Major Update to Google BigQuery

Google Showcases F1

BlinkDB Alpha of Approximate Query Engine Released 

NuoDB Challenges NoSQL

Google's F1 - Scalable Alternative to MySQL


To be informed about new articles on I Programmer, install the I Programmer Toolbar, subscribe to the RSS feed, follow us on, Twitter, Facebook, Google+ or Linkedin,  or sign up for our weekly newsletter.




or email your comment to:

Banner Five Years On

It is 5 years since first proclaimed its message "Every student in school should have the opportunity to learn to code". Its Hour of Code event has seen exponential growth year [ ... ]

Databricks Adds ML Model Export

Databricks has added a machine learning model export feature that can be used to export models from Apache Spark MLib.

More News

Last Updated ( Thursday, 21 November 2013 )

RSS feed of news items only
I Programmer News
Copyright © 2018 All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.