swecrets

ClickHouse's Origins and Where It Fits

Foundations — Rung 111 min read

Learning Objectives

  • Explain the specific problem Yandex.Metrica needed ClickHouse to solve, and why off-the-shelf databases at the time couldn't solve it
  • Explain what the name "ClickHouse" signals about the system's original design intent
  • Place ClickHouse correctly within the warehouse / lake / real-time-OLAP landscape, rather than assuming its name means "data warehouse"
  • Judge, from a workload description, whether it's shaped like the problem ClickHouse was built to solve

Why This Matters

Every tool has a shape it was forged around, and that shape doesn't disappear just because the tool later becomes popular. Teams reach for ClickHouse because they've heard it's fast, deploy it against a workload that doesn't resemble the one it was built for, and then spend months fighting a database that was never trying to be good at what they're asking of it. Knowing the actual problem ClickHouse was built to solve — not the marketing summary, the real originating problem — is what lets you predict, before you write a line of schema, whether it's going to be a great fit or a slow-motion mistake.

Analogy

Picture a regional delivery company with one route that's brutal: a steep, washed-out mountain road that no standard delivery van can climb reliably in bad weather. Nothing off the shelf is built for it, so the company designs and welds together its own truck — reinforced suspension, extra ground clearance, a low gear ratio for climbing loose gravel at a crawl.

It works so well that other companies serving similarly rough, unpaved terrain start asking to buy the design. Word spreads, and soon it's not "the mountain company's truck" anymore — it's just a truck you can buy.

But a courier service that only ever drives flat city blocks, stopping every thirty seconds to hand off a package? That reinforced mountain truck is overbuilt for them — heavier, less nimble in traffic, slower to accelerate away from a stop. It wasn't designed for their terrain. It was designed for someone else's.

A search-analytics company's brutal terrain

ClickHouse's mountain road was Yandex.Metrica, a web-analytics product built by Yandex (at the time, the dominant search engine in Russia) — the same category of product as Google Analytics. In 2009, engineer Alexey Milovidov and colleagues at Yandex set out to test a specific hypothesis: could you generate analytical reports in real time, straight from raw, non-aggregated data that was still arriving? Three years later, in 2012, the resulting engine went into production to power Yandex.Metrica. Metrica's customers didn't just want a handful of pre-built charts. They wanted to ask arbitrary questions of their raw traffic data — "sessions from mobile Safari, in this region, on this campaign, over these three weeks" — and get an answer in seconds, over data that hadn't been pre-aggregated for that specific question in advance, at a volume of billions of raw events.

That combination is what made the problem brutal. A database can usually give you two of "huge volume," "fast," and "flexible ad hoc queries over raw (not pre-aggregated) data" — but the standard tools of the time each gave up on one of the three. Row-oriented databases, built to fetch a single record quickly, had to scan and reconstruct entire rows even for a query that only needed three columns out of thirty, which fell over at Metrica's volume. Pre-built OLAP cubes could answer questions fast, but only the questions someone had anticipated and pre-aggregated for — a new slice meant waiting on a rebuild, which killed the "arbitrary question" requirement. Yandex needed all three properties at once, and nothing on the market delivered them together. So the team built its own storage engine, from scratch, shaped entirely around that one workload.

That engine was used internally for four more years before Yandex open-sourced it in 2016 under the Apache 2.0 license, making it available to any company that recognized the same shape of problem in its own traffic.

What the name tells you

The name really is a portmanteau of "Clickstream" and "Data Warehouse" — Yandex's own naming FAQ says so directly: ClickHouse was supposed to keep records of clicks from people all over the internet. That's a trap, though, if you take it too literally. The name describes the use case it was built for — recording and reporting on clickstream data — not the category of database it turned out to be. Despite "Data Warehouse" sitting right there in the name's own etymology, ClickHouse doesn't behave like the batch, scheduled-load warehouses from the previous lesson. It sits in the real-time OLAP category: continuous ingestion, data queryable within seconds, built to serve live dashboards and applications — the newsstand from the last lesson's analogy, not the library.

Metrica needs fast,flexible, huge-scalead hoc reportsExisting tools eachgive up one of thethree propertiesYandex builds a customengine shaped aroundthat exact workloadOpen-sourced;similarly-shapedworkloads adopt itCompanies without that workload shape often find it a poor fit —the same reinforced truck that wins on a mountain roadis overbuilt and slow in city traffic.
From one company's mountain road to a general-purpose tool. A horizontal timeline with four labeled points connected by a line, left to right. Point one: 'Yandex.Metrica needs ad hoc reports over billions of raw click events, fast.' Point two: 'Off-the-shelf row stores and OLAP cubes each fail on one of speed, scale, or flexibility.' Point three: 'Yandex builds a custom storage engine internally, shaped around exactly that workload.' Point four: 'Engine is open-sourced; companies with a similarly-shaped workload elsewhere adopt it.' A small side note below point four reads: 'Companies without that workload shape often find it a poor fit — same as the mountain truck in a city.'

A worked example

Companies that later adopted ClickHouse tend to share Metrica's terrain, not just its name recognition. A content-delivery network fielding an enormous volume of HTTP request logs, whose customers want to slice their own traffic by status code, country, and endpoint on demand, is asking the same three-part question Metrica asked: huge volume, fast answers, arbitrary slices over raw events. That's why ClickHouse shows up repeatedly in exactly this kind of company — not because it's generically "the fast one," but because the workload shape matches.

Contrast that with a team building an inventory system where every warehouse scan needs to immediately and correctly decrement a specific item's stock count. That's a single-row, transactionally consistent update — the courier's flat city block, not the mountain road. ClickHouse's design didn't optimize for that problem, because that wasn't the problem it was born solving.

Common Misconception

"ClickHouse" literally comes from "Data Warehouse," so it must be a data warehouse in the Snowflake / BigQuery sense — batch-loaded, scheduled, built for deep historical exploration."

The etymology is real — the name does combine "Clickstream" and "Data Warehouse." But a name is a snapshot of what a system's creators called it, not a binding claim about which category it landed in. ClickHouse was built for the opposite end of the freshness spectrum from a batch warehouse: continuous ingestion, with data queryable within seconds, not hours. It belongs in the real-time OLAP category alongside Druid and Pinot, not alongside Snowflake and BigQuery — even though its own name, taken literally, points the other way.

Predict, Then Verify

For each workload, predict: is this shaped like the problem ClickHouse was originally built to solve at Yandex.Metrica (yes or no), and why?

  1. An ad-tech platform logging billions of impression events per day, where analysts run varied, unplanned ad hoc queries — slicing by creative, campaign, geography, device — with new slices requested every week.
  2. A hospital's patient-records system, where each visit updates one patient's record and every write must be immediately durable and consistent before the next step in care can proceed.
  3. An observability platform ingesting metrics and logs from thousands of servers continuously, where engineers run ad hoc queries mid-incident to hunt for anomalies as fast as possible.
Reveal the answer →
  1. Yes. Huge volume of raw events, ad hoc slicing that wasn't planned in advance, need for fast answers — this is Metrica's workload shape almost exactly.
  2. No. This is small-volume, single-row, transactionally consistent — the opposite shape. It needs a system optimized for correct, immediate individual updates, not fast scans over huge raw event volumes.
  3. Yes. Continuous high-volume ingestion plus fast, unplanned ad hoc queries under time pressure is the same shape of problem, just in an operations context instead of a marketing one.

Summary

ClickHouse wasn't built as a general-purpose database that happened to get fast — it was built inside Yandex to solve one specific, brutal problem: giving Yandex.Metrica's users fast, ad hoc, unplanned queries over billions of raw click events, a combination of scale, speed, and flexibility that the row stores and pre-built OLAP cubes of the time couldn't deliver together. The name reflects that origin — a portmanteau of "Clickstream" and "Data Warehouse" — but that etymology describes the use case it was built for, not the category it landed in: ClickHouse belongs in the real-time OLAP category from the last lesson, not alongside the batch-loaded warehouses its own name might suggest. You can now use that origin story as a test: when a workload's volume, freshness needs, and query flexibility resemble Metrica's, ClickHouse tends to fit. When it looks like small-volume, single-row, transactionally consistent work instead, it doesn't — no matter how fast ClickHouse's reputation says it is.

Quiz

  1. 1. Why couldn't Yandex just use an off-the-shelf database for Yandex.Metrica's reporting problem?

  2. 2. What does the name "ClickHouse" actually signal about the system, and what's a common mistake in reading it?

  3. 3. A learner says: "ClickHouse is fast, so it must be a good fit for any workload that needs speed." What's the flaw in that reasoning?

  4. 4. Which of these workloads looks most like the problem ClickHouse was originally built to solve at Yandex.Metrica?

  5. 5. A team building a small e-commerce checkout service — a few hundred orders per hour, each needing an immediate, consistent balance update — is considering ClickHouse because "it's the fastest analytics database out there." Should they use it?