Agentic AI & GenAILightning talk20min
What It Costs to Put an LLM in Your Query Plan
This deep-dive explains ClickHouse AI functions and inference, showing how LLM calls inside SQL impact cost, latency, and query planning. It presents production benchmarks from single-row to million-row scale, explains the design choices, where the approach excels, and how to avoid slowing down the database.
talk.summaryAiDisclaimer
Pete HamptonClickHouse
An LLM call inside a SQL function is a row operation that takes seconds, costs money, and can't be indexed - query planners were never built for that. This is the engineering deep-dive on ClickHouse's AI functions and inference that powers them. We start with one row, then a million: real throughput and latency numbers from production benchmarks, showing what changes when a seconds-long, unindexable operation hits the query planner at scale. You'll see why they were designed the way they were, where the pattern wins outright, and what we are doing to make sure we don't turn the fastest database in the world into the slowest.