What it does
Xinference lets you run many open-source AI models on your own machines, from a laptop to a data centre. It offers an API (a doorway that lets programs talk to it) that works like OpenAI's, so apps built for OpenAI can switch to your own models.
It supports language, speech, image and multimodal models, and can use several different engines underneath. It can spread work across several computers.
Besides the free open-source version, there is a paid Enterprise edition and a hosted cloud service.
Who it's for
Developers and teams who want to host open AI models themselves instead of paying per use for a cloud API.
Can I use it without coding?
With some setup. You install it with Python or Docker and manage it from the command line. New to this? Start here.
Can I run it?
- Cost
- Free, with optional paid extrasThe open-source version is free; the makers also sell a hosted or pro version.
- Runs on
- Server, Cloud
- Privacy
- Your choice: run it on your device or use an online service
- API key
- Not needed
Try it
- Install Xinference with Python's package installer or with Docker.
- Start it and choose a built-in model to launch, then connect your apps to it.
Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?
Before you install
- Only download from the official GitHub page. Copies with similar names exist for many popular projects.
- License (Apache-2.0): Permissive: free to use, change and share, including for business.
- More on staying safe