理解MonetDB about column-oriented database

• Overview

  MonetDB is designed from scratch focusing on handling analytical workloads efficiently on modern hardware.

• Column-at-a-time

  MonetDB stores data one column-at-a-time both in memory and on disk and exploits bulk processing and late materialization.

• MMP

  It solely relies on memory mapped files avoiding the overhead and complexity of managing a buffer pool.

  理解buffer pool in computer

• Differ from traditional RDBMS architecture

  • Execution engine, which uses a column at-a-time-algebra

    Traditional query execution uses a tuple-at-a-time, pull-based, iterator approach in which each operator gets the next input tuple by calling the next() method of the operators of its children in the operator tree.

    Well supported by compiler technology to extract maximum performance from CPUs through techniques such as strength reduction, array blocking, loop piplining.

    The column-at-a-time processing is realized through the BAT Algebra which stands for Binary Association Table.

    BATs are essentially in-memory (or memory mapped) arrays.

    The reason behind the efficiency of the BAT Algebra is its hardcoded semantics, causing all operators to be predicate-less. which in other words is “zero degress of freedom”

    The philosophy behind the BAT Algebra can also be paraphrased as “the RISC approach to database query languages”: by making the algebra simple, the opportunities are created for implementations that execute the common case very fast.

  • Processing algorithms, that minimize CPU cache misses rather than IOs

  • Indexing, which is not a DBA task but happens as a by-product of query execution

  • Query optimization, which is done at run-time, during query incremental execution

  • Transaction management, which is implemented using explicit additional tables and algebraic operations, so read-only workloads can omit these and avoid all transaction overhead