Plain-AyI
← All projects
Run AI on your computer

PowerInfer

Runs large AI language models faster on a home computer with a single ordinary graphics card.

Research-backedNew this week

What it does

PowerInfer is an engine for running large language models on a personal computer. It splits the work between the graphics card (GPU) and the main processor (CPU) in a clever way.

It relies on the fact that only a small set of a model's parts are used most of the time. Those are kept on the graphics card, while the rest run on the processor, which reduces how much graphics memory is needed. Its makers say this is much faster than similar tools on the same hardware.

It works with specific models prepared for this method. It has been tested on Linux and Windows, and on Apple Silicon Macs without much speed gain.

Who it's for

Technical users and researchers who want to run big AI models locally on consumer hardware.

Can I use it without coding?

Developers only. You build it from source code with developer tools and download special model files. New to this? Start here.

Can I run it?

Cost
FreeNothing to pay or sign up for.
Runs on
Linux, Windows, Mac
Computer
Works best on a powerful computer or one with a graphics card (GPU)
Privacy
Runs on your own device: your data stays with you
API key
Not needed

Try it

Using Claude Code, Cursor or another AI coding assistant? Paste the prompt and it will walk you through the setup.New to this?

Before you install

  • Only download from the official GitHub page. Copies with similar names exist for many popular projects.
  • License (MIT): Very permissive: free to use, change and share, including for business.
  • More on staying safe

Where people are talking about it