Flashcache

Not to be confused with NetApp Flash Cache, IBM FlashCache, or Oracle Database Flash Cache.

Flashcache is a disk cache component for the Linux kernel, initially developed by Facebook since April 2010, and released as open source in 2011. Since January 2013, there is a fork of Flashcache, named EnhanceIO and developed by sTec, Inc. Since 2015 that fork became unmaintained and it was forked again and maintained by individuals.

Flashcache works by using flash memory, a USB flash drive, SD card, CompactFlash or any kind of portable flash mass storage system as a write-back persistent cache. An internal SSD can also be used for increasing performance.

Overview

Using flash memory (NAND memory devices) for caching allows Linux kernel to service random disk IO with better performance than without the cache. This caching applies to all disk content, not just the page file or system binaries. Flash memory based devices are usually a magnitude faster than spinning HDDs for random IO, but with less advantage or even slower in sequential read/writes. By default, flashcache caches all full blocksize IOs, but can be configured to only cache random IO whilst ignoring sequential IO.

Similar technology exists in Microsoft Windows as ReadyBoost since Windows Vista.

Implementation

Flashcache is built on top of the Linux kernel's device mapper. The data structure of the cache is a set-associative hash table, in which the cache is divided up into a number of fixed-size sets (buckets), using linear probing within a set to find blocks. The device mapper layer breaks up all I/O requests into blocksize chunks before passing the requests to the cache layer.

When a write request happens, the corresponding cache block is marked dirty; dirty cache blocks are written lazily to disk in the background. There are a few parameters to control the write-back policy: dirty-threshold, idleness and contiguity with other dirty blocks about to be written back.

Limitations

There are a few limitations, imposed by the implementation of flashcache:

Atomicity

Cache block writes are currently non-atomic.

TRIM support

ATA TRIM command to optimize flash memory are not yet supported.

Cache pollution protection

A process can be marked non-cacheable to prevent flashcache cache its requests; however, if a process that marked itself non-cacheable dies, flashcache has no way of cleaning up.

Alignment

Relying on the device mapper resulted in caching performance issues and no caching of writes that are not multiple of 4 KiB. Primarily, this affects the Xen hypervisor. Thus, EnhanceIO has moved away from the device mapper integration, yielding higher performance for unoptimal use cases.

Write-around read latency impact

in write-around mode all writes bypass the cache for high consistency. The current implementation will fetch reads through the SSD device and then deliver them to the actual reader. This means that previously uncached blocks will always need to go to the SSD device first, causing a constant write IO. Not an issue on enterprise SSD or highend PCIe devices as facebook uses, but degrades performance on lower end SSD.

Write-around read cache warm-up phase

in write-around mode FlashCache has no information to compare the age of cached pages over the on-disk ones. (1) Because the device could have been mounted outside of FlashCache (2) Because no writes are tracked in this mode. This results in an empty cache after each volume activation (i.e.: reboot). Performance will be degraded until all hot areas have been cached.

See also

• bcache
• dm-cache
• Cache Acceleration Software (Intel's product)

External links

• Performance Comparison among EnhanceIO, bcache and dm-cache (LKML)
• EnhanceIO, Bcache & DM-Cache Benchmarked
• Flashcache at Facebook: From 2010 to 2013 and beyond
• Facebook Releases Flashcache 3.0 Archived 2013-12-20 at the Wayback Machine