The Praxis proxy or: How I Learned to Stop Worrying and Love AI workloads

Every proxy you run today was designed for a world where routing decisions live in paths, hosts, and headers. AI traffic broke that assumption. Which model? Which provider? Is this an MCP call or a chat completion? The answers are buried in JSON bodies - invisible to nginx, awkward in Envoy, and increasingly the difference between a demo and a production system.
This talk introduces Praxis, an open-source proxy framework built for AI infrastructure. We'll look at what makes AI workloads genuinely different - body-based routing, credential isolation, streaming-first traffic, token-denominated cost, and agent protocols like MCP - and why bolting these onto existing proxies keeps failing. Then we'll dig into why Praxis is written in Rust: zero unsafe code and Rustls in the most exposed component of your stack, predictable latency with no GC jitter, and the ability to stand on Cloudflare's battle-tested Pingora rather than reimplementing HTTP.
Crucially, Praxis is a framework for building proxies, not just a proxy. Everything is a filter - routing, rate limiting, guardrails, model selection - and your custom filters use the same traits as the built-ins, compiling straight into the binary. The same primitives compose into an AI gateway, an ingress, an egress proxy, or a mesh sidecar. Whatever your use case, you can assemble it from config in minutes - and when no filter fits, write your own and make the proxy truly yours.