NVIDIA GPUs
The standard hardware for training and serving AI models. NVIDIA GPUs are the most widely used accelerators for training and running machine learning models, supported by the CUDA software stack that almost every AI framework builds on. They range from data-centre cards to workstation and consumer GPUs, and are available on all major clouds. On Kubernetes, NVIDIA's device plugin and GPU Operator make them schedulable resources.
When it is the right choice
- You self-host language, vision or speech models with real traffic
- You fine-tune models on your own data
- You need predictable latency that shared hosted APIs cannot guarantee
- Not worth it for occasional or low-volume use; a hosted API is usually cheaper than idle GPUs
How we use it
- Size GPU memory and count from your model, context length and traffic, not from guesswork
- Provision GPU node pools with Terraform and the NVIDIA GPU Operator on Kubernetes
- Monitor utilisation, memory and temperature with DCGM metrics in Prometheus and Grafana
- Review utilisation regularly and right-size or schedule capacity to cut idle spend
Kubernetes
vLLM
Prometheus
TerraformWhere it fits in our work
Related articles
GPU Nodes on Kubernetes Without Wasting MoneyNode pools, MIG, time-slicing, scale to zero and spot instances: practical ways to keep Kubernetes GPU costs under control for AI workloads.
Running Open Models in Production: A Practical Starting PointWhat it takes to run Llama, Mistral, Qwen, Gemma or DeepSeek in production: picking a model, sizing hardware, serving it and keeping it healthy.NVIDIA GPUs: frequently asked
Should we rent cloud GPUs or buy our own?
Cloud GPUs suit variable or early-stage workloads. Owning hardware can pay off for steady, high use, but adds power, cooling and maintenance; we model both with your expected usage.
How do we know if our GPUs are underused?
Export GPU metrics with NVIDIA DCGM to Prometheus and track utilisation and memory per workload. Idle capacity then shows up on a dashboard instead of only on the bill.
Running NVIDIA GPUs, or deciding whether to? Tell us what you have; the first consultation is free.
Let's talk »
