DevDay Pune

⚠️ ATTENTION ⚠️
The event has been postponed to 25th July, 2026. Kindly take a note.

🎟️ Please RSVP to join-in for this conversation in-person.

The Product Outgrew the Database, So We Moved to ClickHouse

For years, our analytics platform served a workflow nobody complained about. We build planning software for out-of-home advertising — the billboards and digital screens you pass on roads, in transit, in malls. Our users are media planners: they assemble a campaign of screens, kick off an insights run, wait, and review. Behind that sat Postgres for analytics, MongoDB and Redis for proximity search, and a layer of services stitching results across the seams. Some queries took 22 seconds, some took eight minutes. Nobody minded as nobody was watching them run.

Then the product changed shape, for a reason that had nothing to do with database performance. Planners were doing their actual planning work in an external tool, which had an interactive map where filters update continuously as you drag them, and only arrive at our platform to push the execute button.

So we reimagined it: a conversational, AI-driven planner where users describe what they want in natural language and watch a live map, KPI panel, and data grid update together. That surfaced three challenges. The map had to feel responsive, sub-2-second responses, with many planners querying at once. Filtering by points of interest, “billboards near any Starbucks in California”, became the default way to plan, not a power-user feature. And an AI agent now sits between the user and the data: most requests flow through a curated set of APIs the agent calls with structured parameters, but anything those APIs can’t answer falls through to a second agent that queries the database directly.

The fix wasn’t another cache layer or a bigger cluster. It was rebuilding around ClickHouse, a single columnar engine fast enough on raw data that you don’t need to anticipate the next question. Put spatial indexing inside that same engine, and “billboards near any Starbucks” never has to leave the database to be answered. This talk is the postmortem: what the product change demanded of the data layer, what we tried that didn’t work, and the patterns that travel well beyond ad-tech.

Takeaways

• Database choices are downstream of product shape. We didn’t migrate to ClickHouse for performance, so we migrated because the product became an interactive, conversational map, and the old workflow (assemble → wait → review) no longer existed. Re-evaluate your data layer when the interaction model changes, not just when latency complaints arrive.
• “Nobody minded 8-minute queries” until someone was watching them run. Latency is only a problem relative to the interaction. The same query that was fine in a batch workflow is fatal in a live-filtering one. Know which world you’re in.
• A columnar engine fast enough on raw data buys you the questions you didn’t anticipate. Caches and pre-aggregation optimize for known queries. When an AI agent (or a user dragging filters) can ask anything, raw speed beats anticipation.
• Push spatial logic into the engine, not across services. “Frames near any Starbucks in California” stayed slow as long as the answer had to leave the database. Co-locating spatial indexing with the data removed the seams between multiple data stores — and the services stitching them.

Prerequisites
Comfortable with databases and SQL in a production setting, and you’ve hit a performance wall at least once.