What it does
KServe is a platform for running AI models in production on Kubernetes, the system for managing software across many servers.
It supports both generative models, such as large language models, and traditional prediction models from many machine learning frameworks, with one standard way to send requests.
It offers features like automatic scaling, including scaling down to nothing when idle to save money, and gradual rollouts of new versions.
Who it's for
Engineering teams serving AI models at scale on Kubernetes.
Can I use it without coding?
Developers only. It runs on Kubernetes and is set up by infrastructure engineers. New to this? Start here.
Can I run it?
- Cost
- FreeNothing to pay or sign up for.
- Runs on
- Cloud, Server
- Computer
- Made for servers or data centres
- Privacy
- Runs on your own device: your data stays with you
- API key
- Not needed
Try it
Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?
Before you install
- Only download from the official GitHub page. Copies with similar names exist for many popular projects.
- License (Apache-2.0): Permissive: free to use, change and share, including for business.
- More on staying safe