Package: libmmmulti-dev (0.1-2)

header only library for mmmulti

Sometimes you have a lot of plain-old data, but you need random access to it. These header-only classes combine memory-mapped files with high-performance parallel sorting and appropriate indexing strategies to support very large (>memory but <disk) multimaps, multisets, and interval trees.

This implements a memory backed multimap intended for use where:

 * your keys are integers, or can be mapped to dense range of integers,
 * the memory mapped file is on fast storage, like an SSD (although
   this is not a requirement),
 * you have arbitrary values of fixed size (e.g. structs, other POD
   types) that can be sorted,
 * you don't need dynamic updates of the table,
 * and you are likely to run out of memory of you use a traditional
   map or hash table,
 * but you can handle approximately 1 bit per record in RAM.

These may seem to be very specific, but many problems can be mapped into a dense integer set. mmmulti::map developed first as a data structure to support seqwish, which uses it to generate precise variation graphs from pairwise alignments between collections of sequences. As this multimap forms a key data processing kernel in the algorithm, it can scale to extremely large problem sizes, limited only by available disk space. Although performance is much slower than an in-memory structure, it is virtually guaranteed to be able to complete the compute.

Tags: Software Development: Függvény-könyvtárak, Role: Development Library

This page is also available in the following languages (How to set the default document language):