Plain-AyI
← All projects
Running AI at scale

LMDeploy

A toolkit for compressing AI language models and serving them quickly on your own GPUs.

Research-backedNew this week

What it does

LMDeploy helps teams run large language models efficiently. It can compress models (for example to 4-bit) so they use less memory, and it serves them through a fast engine so many people can use them at once.

Its makers report speed gains over other serving tools and support for many popular open models, including image-and-text models. It works with NVIDIA graphics cards and some other AI chips.

Who it's for

Engineers deploying open AI models in production.

Can I use it without coding?

Developers only. It is installed with Python and needs a GPU server. New to this? Start here.

Can I run it?

Cost
FreeNothing to pay or sign up for.
Runs on
Server, Linux
Computer
Made for servers or data centres
Privacy
Runs on your own device: your data stays with you
API key
Not needed

Try it

Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?

Before you install

  • Only download from the official GitHub page. Copies with similar names exist for many popular projects.
  • License (Apache-2.0): Permissive: free to use, change and share, including for business.
  • More on staying safe

Where people are talking about it