Llama
Meta's family of open-weight large language models. Llama is a family of large language models from Meta whose weights can be downloaded and run on your own hardware. The family spans small models that fit on a single GPU up to large models that need a multi-GPU server, with instruction-tuned variants for chat and tool use. It is used for private chat assistants, retrieval-augmented generation, summarisation and fine-tuning on company data.
When it is the right choice
- You need data to stay inside your own cloud account or data centre
- You want to fine-tune a capable base model on your own domain data
- Steady, high-volume traffic makes owned GPUs cheaper than per-token pricing
- Not ideal when you need the strongest reasoning available today with no GPU operations; a hosted frontier model is simpler
How we use it
- Serve Llama with vLLM or Ollama on GPU nodes we provision with Terraform
- Run evaluation suites on your own prompts before and after every model or quantisation change
- Trace latency, token usage and error rates with OpenTelemetry and Grafana dashboards
- Check Meta's licence terms for your use case as part of the design work
vLLM
Ollama
NVIDIA GPUs
Hugging FaceWhere it fits in our work
Related articles
From Idea to AI Feature in Six WeeksHow an AI project runs end to end in six weeks: discovery, design, prototype, infrastructure, rollout, then measuring and improving.
CI/CD for Models and PromptsEvaluation gates, model registries, canary releases and rollback: applying CI/CD discipline to prompts and models in LLM applications.
Fine-Tuning or Prompting? A Decision Guide for Small TeamsWhen prompting, RAG or fine-tuning is the right way to adapt a language model, and what each one costs a small engineering team.Llama: frequently asked
Is Llama open source?
Llama weights are openly available, but they are released under Meta's own licence with conditions, not a standard open-source licence. Read the licence for the specific model version before you build on it.
What hardware do we need to run Llama?
Small Llama models run on a single modern GPU, and quantised versions can run on a workstation. Larger models need several data-centre GPUs; we size this against your expected traffic and latency targets.
Running Llama, or deciding whether to? Tell us what you have; the first consultation is free.
Let's talk »
