What it does
llama-swap sits in front of local AI servers such as llama.cpp and vLLM. When an app asks for a model, it starts that model and stops others, so you can use many models on one machine.
It speaks the OpenAI and Anthropic API styles, so many apps can connect to it. It also supports image generation, speech and transcription servers, and has a web page for logs and status.
Its makers say it is one program and one settings file, with no other dependencies.
Who it's for
People running several local AI models who want apps to switch between them smoothly.
Can I use it without coding?
With some setup. It installs with Docker or a package manager and is set up with a settings file. New to this? Start here.
Can I run it?
- Cost
- FreeNothing to pay or sign up for.
- Runs on
- Mac, Linux, Windows
- Privacy
- Runs on your own device: your data stays with you
- API key
- Not needed
Try it
- Install llama-swap using Docker or your computer's package manager.
- Write a settings file listing your local models.
- Point your AI apps at llama-swap.
Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?
Before you install
- Only download from the official GitHub page. Copies with similar names exist for many popular projects.
- License (MIT): Very permissive: free to use, change and share, including for business.
- More on staying safe