Beyond the Pod: Architecting and Scaling AI Agents on Kubernetes

As we shift from simple LLM wrappers to autonomous AI agents, a critical question emerges: Where and how should they run? On Kubernetes, our instinct is to treat agents like traditional microservices, giving each its own Pod for isolation and identity. But AI agents aren’t microservices.

Unlike long-running APIs, agents are ephemeral, bursty, and highly stateful. They might spin up for a task, spawn parallel sub-agents, and then sit idle for hours awaiting human approval. Keeping a dedicated Pod alive for every agent is massively wasteful.

In this talk, we’ll explore Agent Substrate, an open-source system on Kubernetes that decouples the logical agent ("Actor") from the physical execution environment ("Worker" Pod).

What you will learn:

• Heavy Multiplexing: Map thousands of stateful, idle AI agents onto a small pool of shared Kubernetes Pods.
• Hibernate and Teleport: Look under the hood at how Substrate uses OCI containers and gVisor to achieve sub-second activation, session teleportation, and perfect state persistence across hibernation.
• Rethink Identity & Security: Shift network policies and access controls away from the Pod level and attach them dynamically to lightweight execution units.

Leave this session with a fresh perspective on how to isolate and scale the next generation of AI applications without abandoning the Kubernetes ecosystem you already know.