Plain-AyI
← All projects
Running AI at scale

OpenLLM

Run open AI models on your own servers as a ready-to-use service that works like OpenAI's.

New this week

What it does

OpenLLM lets developers take open AI language models, such as Llama, Qwen or DeepSeek, and run them as a service on their own hardware. Other programs can then talk to the model the same way they would talk to OpenAI's service.

It includes a simple built-in chat page for trying the model. The README lists how much graphics-card memory each model needs; many need large GPUs.

It can also deploy models to BentoCloud, its makers' hosted AI platform, for teams that don't want to manage servers.

Who it's for

Developers and teams who want to host open AI models themselves.

Can I use it without coding?

Developers only. It is installed with Python and run from the command line, usually on a GPU machine. New to this? Start here.

Can I run it?

Cost
FreeNothing to pay or sign up for.
Runs on
Server, Cloud
Computer
Works best on a powerful computer or one with a graphics card (GPU)
Privacy
Your choice: run it on your device or use an online service
API key
Not needed

Try it

Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?

Before you install

  • Only download from the official GitHub page. Copies with similar names exist for many popular projects.
  • License (Apache-2.0): Permissive: free to use, change and share, including for business.
  • More on staying safe

Where people are talking about it