Kubernetes
The standard platform for running containers at scale. Kubernetes is an open-source container orchestration platform originally designed at Google and now maintained under the Cloud Native Computing Foundation. It schedules containers across a cluster of machines, restarts failed workloads, handles service discovery and rolling updates, and scales applications. It is available as a managed service on every major cloud and can run on your own hardware.
When it is the right choice
- You run many services that need consistent deployment, scaling and recovery
- You need to share GPU and compute capacity across teams and workloads
- You want the same platform across clouds or on your own servers
- Not the right choice for one or two simple services; a managed container service or a plain VM costs less to run
How we use it
- Build clusters with Terraform on EKS, GKE, AKS or bare metal, with GitOps through Argo CD
- Set up monitoring, logging and alerting with Prometheus and Grafana from the first day
- Use AI-assisted runbooks to triage alerts and propose fixes for a human to approve
- Plan and test version upgrades on a schedule so clusters never fall behind support
Where it fits in our work
Related articles
CI/CD for Models and PromptsEvaluation gates, model registries, canary releases and rollback: applying CI/CD discipline to prompts and models in LLM applications.
GPU Nodes on Kubernetes Without Wasting MoneyNode pools, MIG, time-slicing, scale to zero and spot instances: practical ways to keep Kubernetes GPU costs under control for AI workloads.
vLLM vs Ollama vs TGI: Choosing an Inference ServerHow vLLM, Ollama and Hugging Face TGI differ in throughput, hardware support and operations, and which one fits your workload.Kubernetes: frequently asked
Do we need Kubernetes?
Only if the operational benefits outweigh the effort. For a small number of services, Amazon ECS, Cloud Run or a well-managed VM is often simpler; we recommend what fits.
Managed Kubernetes or self-managed?
Managed services such as EKS, GKE and AKS take the control plane off your hands and suit most teams. Self-managed makes sense on your own hardware or with special requirements.
Running Kubernetes, or deciding whether to? Tell us what you have; the first consultation is free.
Let's talk »

