Unlock this content
Enter your email to unlock this content for free
Scaling Ingestion
TL;DR
ClickHouse is designed for bulk inserts, not row-by-row streaming. Understanding this fundamental design choice helps you scale writes effectively. The ingestion proxy pattern addresses write scaling challenges by introducing a buffer layer between your applications and ClickHouse.
How Writes Work
ClickHouse is designed for bulk inserts, not row-by-row streaming. Understanding this fundamental design choice helps you scale writes effectively.
How writes work
- Blocks: Data arrives in blocks (typically 1M rows or 256MB)
- Parts: Blocks are written as immutable parts on disk
- Merges: Background jobs compact small parts into larger ones
- Budget: Merges consume CPU + disk + memory + network bandwidth
Loading diagram...