Backgroud flush and compaction

MaybeScheduleFlushOrCompaction

MaybeScheduleFlushOrCompaction会使用线程池调度,最后在后台线程中调用BackgroundFlushBackgrondCompaction分别做memtable的flush和ssfile的compaction.

MaybeScheduleFlushOrCompaction

后台线程调度Schedule

schedule-bgtread

后台线程flush

生成flushRequest放入flush队列中

flush_queue_中放入FlushRequest的数据流程如下:

flush_queue_put

具体函数调用细节如下:

flush queue put detail

后台线程处理flush队列中请求

后台线程执行BackgroundFlushflush_queue_中取出FlushRequest转换为FlushJob.

flush-data-flow-overview

cfd会被flush的条件

bool MemTableList::IsFlushPending() const {
  if ((flush_requested_ && num_flush_not_started_ > 0) ||
      (num_flush_not_started_ >= min_write_buffer_number_to_merge_)) {
    assert(imm_flush_needed.load(std::memory_order_relaxed));
    return true;
  }
  return false;
}

最终调用WriteLevel0Table 将memtable写入磁盘中,具体调用关系如下:

backgroud-flush

后台线程compact

cfd放入compact队列

background-compaction-put

处理compact队列,生成compactionJob

后台线程会通过PickCompactionFromQueuecompaction_queue_中取出需要compact的ColumnFamilyData, 然后调用ComlumnFamilyData的PickCompaction 选择compactio的input level, output leve, 以及input files等,

backgroup-compaction

多线程并发compact

在compact Prepare中会将compactJob划分为不同的SubCompactionState,然后由多线程并发执行压缩

background-compaction-job

Compaction Picker

三种compaction style

Level Style Compaction

Universal Style Compaction

FIFO Style Compaction