Day 11/100

Day 11/100

#Scylla Operations Course [Part 4 - Scylla Monitoring]

Disk Scheduling

Screenshot 2022-03-31 at 11.57.21 PM.png

sstable

  • composed of multiple parts, all sharing similar structure "mc--big-"
  • TOC- text file for which the components exists for this specific sstable "generation"
  • Bloom Filter- provides answer if partition is not stored in sstable
  • Statistics - stores different stats about the file - including minimal values used for delta encoding
  • Summery - Lookup into the index file "pages" that may contain the partition
  • Index - Lookup into the data file "blocks" that contain the partition
  • Data- Hold the partition information.
  • CompressionInfor- If compression was enabled provide a map of the compressed files in the data file
  • Scylla - additional scylla meta information + partition sharding information

Screenshot 2022-04-01 at 1.50.28 AM.png

While Reading

Screenshot 2022-04-01 at 1.50.28 AM.png

Single Partition Read path

-Single Page

  • we are reading small buffers
  • Issues read aheads as sequential read requests from the data file are requested
    • Multiple Pages
  • cache the readers for the next page request (querier cache)
  • Eviction based on - resources / memory / time policy

Memtable / Compactions / Streaming / Repair

  • Commitlog - 32 MB files, Pre-Allocating space, recycling commitlogs (for efficiency)

Data Scheduling

Screenshot 2022-04-01 at 2.09.13 AM.png